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ABSTRACT 


A study was conducted to examine various aspects of Distance Learning (DL) 
applications currently under review by the Marine Corps, and determine whether these 
programs, if initiated, provide a positive Return on Investment (ROI). The objective was 
to determine how DL applications may be applied in the most advantageous maimer, to 
increase the overall efficiency of ciurent training programs from both a monetary and 
quality perspective. Specifically, DL applications were evaluated for pertinence to the 
four categories of learners found within the organizational hierarchy. To accomplish this 
objective, information was collected from the DL Branch, Training & Education 
Division, HQMC, as well as from faculty and staff at the Marine Corps Communications 
and Electronics Course, Marine Corps Air Ground Combat Center, 29 Palms, CA. 
Results were favorable with a positive ROI being determined firom the stated 
assumptions. Other findings included that the most beneficial application of DL 
technology should be primarily toward advanced level training with possible 
considerations for Marines awaiting training, and that due to increased instructional 
requirements, the timesavings attributed to advances in training technology should not 
automatically result in reductions in formal course curricula. Simply stated, DL 
technologies provide great value added potential to enhance knowledge transfer in 
today’s dynamic and fluid training environment, but should be viewed primarily as a 
complement to, rather than replacement for, traditional instructional methods. 
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1. INTRODUCTION 


A. OVERVIEW 

1. Background 

In the winter of 1944 the U.S. Army in Europe faced a critical manpower 
shortage, particularly in regard to combat infantrymen. These shortages stemmed from 
the tremendous losses in personnel taken during the proceeding months of campaigning 
in France and Belgium. In an effort to alleviate these shortages, the allied high command 
directed that the flow of replacements from bases in England and the United States be 
substantially increased. To meet this requirement, the quality and length of tr ainin g new 
soldiers received prior to arrival at the front had to be reduced to get them there at a faster 
rate. Replacements were thus sent into combat inadequately trained and as a consequence 
suffered inordinately high casualty rates. Ambrose (1997) discusses how the casualty 
rates of typical replacements exceeded 50 percent, and normally occurred in less than 72 
hours after their amval simply because they lacked basic survival skills. Such 
information, he adds, was relatively easy to pass along and could be taught in a very short 
period of time, if only such time had been made available. Ambrose (1997) blames the 
entire American chain of command for this problem. He states that the plan was 
unquestioningly endorsed by the Army Chief of Staff, General Marshall, and the Supreme 
Allied Commander, General Eisenhower, to muffle complaints from field commanders, 
such as Generals Bradley and Patton, who consistently clamored for more fresh troops. 
All were found to share equal responsibility for casualty rates that are now viewed as 
excessive and wasteful (Ambrose, 1997). 

In pursuit of this objective, it would later be found that many proposals for 
increased training efficiency actually caused more casualties than they helped prevent, as 
critical information was eliminated in the interest of increased throughput. Interestingly 
enough, similar problems would arise again in both Korea and Vietnam as the U.S. Army 
shortened training pipelines to get troops into combat at a faster rate (Ambrose, 1997). 
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2. The Problem 


What relevance do such anecdotes have for today’s military? In the current 
environment of highly complex, technically advanced combat information systems, it is 
imperative that personnel are as proficient as possible at their jobs. To gain and maintain 
such proficiency is becoming exceedingly difficult, however, as the quantity of pertinent 
information is growing at a rate far in excess of the time available to master it. The 
problem is thus how to maximize the potential of training and education methods while 
minimizing the time required for actual knowledge transfer. 

3. A Proposed Solution 

Current training technologies offer a solution to this problem by affording the 
student more opportunity to tailor learning to their individual lifestyles without 
sacrificing the quality of instruction. This paper looks at one aspect of these advanced 
training technologies in the form of distance learning. Distance learning technology has 
already been used with great success in civilian industry, academia, and by government 
agencies, to include the other services. The Marine Corps quickly recognized the 
advantages of such technology and recently invested quite heavily in the supporting 
infrastructure. This paper will attempt to measure the relative return that may be realized 
from such investment, particularly in regard to the more intangible aspects of quality 
instmction and its contribution to unit readiness. 

4. Consequences of Inaction 

Failure to properly address the problem of providing quality instruction in a 
decreased time horizon poses severe consequences for the military. Overall unit 
readiness will obviously suffer with spillover effects that include morale problems due to 
a lack of self-assurance, which may further lead to potential retention and recruitment 
difficulties. Most significantly, however, the military may be doomed to repeat the 
mistakes of the past, as inadequately trained personnel are sent in harm’s way and pay the 
ultimate price for their ignorance. 
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B. PURPOSE 


The purpose of this research is to examine various aspects of Distance Learning 
(DL) applications currently under review by the USMC and determine whether these 
programs, if initiated, provide a positive Return on Investment (ROI). The original goal 
was to evaluate DL as it relates to the following categories of learners found within the 
organizational hierarchy: (1) Initial skills training; (2) Advanced skills training; (3) 
Professional Military Education (PME); and (4) Marines awaiting training 
(MAT)/General Interest. It was quickly discovered, however, that to fully evaluate all the 
above categories in the limited time available was clearly beyond the scope of this 
research, so the focus was reduced to categories one, two, and four with an emphasis on 
categories one and two. Following this evaluation, the feasibility of implementing DL to 
improve the efficiency of current training programs is reviewed through empirical study 
and economic analyses. 


C. THE NATURE OF THE PROBLEM 


1. The Opportunity Cost of Training. 

Where possible, the Marine Corps must employ the full range of world-class 
integrated instructional and information technologies that enhance the full spectrum of 
Marine Corps’ training and education. Such training and education programs must 
encompass a cradle-to-grave methodology that begins with initial entry and specialized 
skills {core and core plus) training and continues through career-long professional 
development. 

Reviews of current training and education processes reveal problem areas that 
significantly impact on operational readiness (Whitbeck, 1999). First, student load 
frequently exceeds formal school seat capacity resulting in large groups of Marines 
Awaiting Training (MAT). Second, initial skill resident training pipelines are 
increasingly long and have increased the training component of the Transients, Trainees, 
Patients and Prisoners (T2P2) manpower account and decreased overall m annin g in the 
operating forces. Additionally, career progression requirements for more senior 
personnel necessitate time away from operating units to attend appropriate grade level 
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schools, which also add to the T2P2 account. The current focus on resident formal school 
attendance as a prerequisite for Military Occupational Specialty (MOS) qualification is 
currently too rigid and frequently contributes to mismatch problems for the Marine Corps 
Reserve (Whitbeck, 1999). Further, the current Marine Corps Institute (MCI) paper- 
based distance learning courses are not closely linked to resident skill training curricula 
and do not contribute to MOS qualification (Taylor, 2000). Training design and 
development is often fragmented, lacking an integrated MOS curriculum. Finally, 
existing non-resident PME courses are based primarily on resident PME curricula, and 
similar to MCI lessons are not optimized for DL, making successful course completion 
difficult (Jones, 1999). 

2. Why Distance Learning? 

To assist the Marine Corps in meeting these challenges with increased flexibility, 
the Marine Corps Combat Development Command’s Training and Education Division 
introduced the Training and Modernization Initiative (TEMI) in FY 1998. The objective 
of this initiative was to maximize the USMC’s limited training design/development and 
training management processes, introducing improved technology into classrooms and 
capitalizing on modem DL technologies. PME programs will also benefit from this 
initiative as current distance education courses are enhanced through the application of 
advanced/emerging information technologies. 

The TEMI attempts to correct many of the current training deficiencies outlined 
above through a comprehensive review and restmcturing of current training and 
education processes. Restmcturing will include improving instmctional design, 
development, delivery and management processes. DL is a major component of the 
TEMI. DL technology has the potential to dramatically change the way in which Marines 
are trained and educated in the future. Just as developments in weaponry have changed 
the face of warfare, investment in DL technology has the potential to transform Marine 
Corps training and education from a centralized, formal school-based, synchronous 
environment to a more distributed, informal, asynchronous climate. The investment in 
technology will be driven by operational readiness requirements, and will be focused on 
improving the efficacy of the training and education programs provided to all Marines. 

As promising as these initiatives sound, however, their implementation comes at a 
price. The objective of this research is to analyze the feasibility of implementing various 




DL technologies within the current infrastructure and determine if a positive ROI can be 
obtained. The analysis addresses cost benefit techniques, cost factors in development and 
cost effectiveness measures. 

3. Why ROI? 

According to Derryberry (1998), during the past several years training 
organizations have been pushed by their clientele - both external and internal - to be 
more accountable for the quality of the resources and services provided. This focus on 
accountability has reached such a level that many training organizations within the 
government, and particularly in the military, are finding themselves competing against 
numerous types of budgetary decisions to maintain their current, if not higher, funding 
levels (Lyau & Pucel, 1995). At the same time, training organizations are being pushed 
to offer more efficient means of accessing performance improvement resources and 
services. The push for accountability means being able to demonstrate that tr ainin g 
interventions have a direct positive impact on improving organizational competitiveness 
while increasing the productivity of individual contributors (Derryberry, 1998). This 
push for greater efficiency and accessibility has understandably led to expectations that 
interactive technologies will play an increasingly significant role in providing both 
services and resources. 

Many organizations are beginning to require inclusion of a coshTienefit analysis 
(CBA) or ROI analysis to justify an expenditure such as what is required for a 
performance improvement environment or electronic learning system. Simply stated, 
ROI methods are used to demonstrate that the value of the benefits realized from using 
such a solution is going to exceed the price of its development and implementation. ROI 
and technology implementation clearly go hand in hand (Derryberry, 1998). 

Even so, the author suggests that ROI is not simply evaluation methodology 
focused solely on financial concerns. Technology decisions are often not just about the 
financial bottom line. Even if data suggest that a low-end non-technological solution 
may yield positive results, there may be a variety of organizational reasons that a low-end 
technology solution my not be as attractive to a decision maker as is the more 
sophisticated, more costly technology solution. While ROI methods provide structure for 
systematically collecting, organizing, and compiling data, the interpretation of those data 
and the resulting perception of value those data represent may be highly subjective. 
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Although difficult to do, a sound ROI strategy on a prospective training technology 
program is certainly worth the effort if it provides the justification necessary to obtain 
funding support. In today’s highly competitive budgetary environment, such a strategy 
may mean the difference between the ultimate success or failure of a program. 

D. RESEARCH QUESTIONS 

1. What are the current DL initiatives currently under review by the USMC? 

2. What are the DL programs currently in use by the USMC? 

3. What is the ROI objective for currently USMC training programs? 

4. What are the costs and benefits of implementing various DL applications to 
current training programs in the USMC? 

5. Do the benefits of DL initiatives as applied to these training programs 
outweigh the associated costs? 

6. What are some common training development cost factors? Should they be 
included in this analysis? 

7. Is Cost-Benefit Analysis (CBA) the most effective method for determining 
ROI on DL? 

8. How cost-effective are current training programs? Has any positive ROI 
already been determined? 

9. What other measures of cost-effectiveness can be used in determining ROI for 
training? 

10. What aspects of quality should be taken into accoimt when determining ROI 
for training? 

E. EXPECTED BENEFITS OF THIS THESIS 

This study will provide information to determine the feasibility, in terms of costs 
and benefits, of various Distance Learning initiatives currently under review by the U.S 
Marine Corps. It will serve as a decision tool by illustrating the potential ROI associated 
with each of these programs, based on the stated assumptions. 
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II. USMC DISTANCE LEARNING PROGRAMS 

A. DISTANCE LEARNING DEFINED 


For the purposes of this paper, the U.S. Marine Corps has defined Distance 
Learning (DL) as: “structured learning that takes place without the physical presence of 
the instructor.” (HQMC T&E DL ROADMAP, 1999). This definition comprises 
traditional paper-based correspondence, but not stand-alone modeling and simulation. 

Key attributes of a sound DL program include (Portaway & Lane, 1992): 

• Physical distance between learner and instructor 

• Program sponsored by an academic institution or functional organization 

• Program is part of a structured curriculum with stated objectives 

• Program provides for two-way communication and feedback between 
institution and learner 

• Program deployed outside the confines of a resident schoolhouse or campus 

• Program includes processes to evaluate learning outcomes 

The traditional classroom may be viewed as an island on which a teacher and a 
group of students, supplied with textbooks and other resources, manage the educational 
process. Occasionally, the students and teachers take trips into the outside world, but the 
overall educational process primarily focuses inward on things happening in the 
classroom. Contemporary telecommunications technology can invert this traditional 
focus. Classrooms now orient to the world outside rather than the immediate 
environment. Instead of remaining isolated islands, classrooms can now be linked by 
communication highways transmitting data (video and audio) to multitudes of remote 
sites. Teachers and students alike may now travel on this new information highway with 
easy access to vast databases and jointly participate in activities that involve other 
students in other countries (Lynton, 1992). This new technology can provide educational 
opportunities to students who lack access to a capable, resident classroom environment. 
Discriminating factors such as geographic dispersion, temporary separation, or 
unavailability of locally qualified instractors need no longer affect the individual’s access 
to a quality education. Distance learning is an exceptional alternative when a tr ainin g 
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facility cannot feasibly or economically provide quality face-to-face instruction to serve 
the remote student’s needs. 

Bramble (1990) states that distance learning is by no means a panacea for all the 
ills of the educational process, nor should it be viewed as a permanent replacement for 
face-to-face instruction. He found that students actually prefer face-to-face instruction. 
Unfortunately, distance learning is often viewed as a conceptual threat to the proponents 
of conventional educational methods. As such, distance learning advocates must be 
careful to present their approach as an alternative to conventional instruction, to be used 
when conventional methods are not feasible or economical. As this study will 
demonstrate, distance learning can be a cost effective alternative, although proponents 
should be cautious not to promote distance education solely on the basis of low cost. To 
properly support this emerging environment, which presently seems to offer great 
potential, organizations must first establish an infrastructure of policies and standards 
focused on long-term strategies for efficient implementation of distance education. 

B. USMC DL TECHNOLOGY STRATEGY 

Distance Learning can be accomplished through a variety of media including 
paper-based instruction, compressed video. Electronic Performance Support Systems 
(EPSS) and Virtual Reality (VR) simulations. As previously mentioned, DL and resident 
courses are viewed as complementary forms of instruction that enable the delivery of the 
right mix of instmctional methods at the right time. In order to successfully transition to 
an effective DL program, however, the organization/institution must fully embrace an 
individual learning model vice merely shifting to a technique of remote teaching. 
Properly implemented, this transition will have wide reaching implications across any 
organization that adopts it, and will require a fundamental shift in traditional approaches 
to training and education. 

The USMC’s technology strategy is centered on the learner. The intent is to 
develop systems that enable current educational systems to reach more Marines with 
better and more focused training and education programs in the near future. In keeping 
with the Commandant’s Planning Guidance (Krulak 1995), the aim of Marine Corps’ 
training and education is to extend learning beyond the boundaries of the traditional 
classroom by exploiting technology. Technology has matured to the point whereby it is 
now possible to deliver high quality interactive instmctional materials using multiple 
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media formats without regard to time, space or distance. Moreover, the rapid growth of 
the Internet has changed the perception of information sharing, electronic commerce, 
methods of instructional delivery and collaborative learning. Academia, government and 
industry are using this medium to train and educate the modem work force. The Marine 
Corps will also use Internet-based technologies to deliver interactive learning products 
and facilitate a collaborative learning environment in the future. 

Distance Learning programs are learner-centered rather than instmctor-centered. 
They are being designed specifically with the distance learner in mind as per the HQMC 
Training & Education DL Roadmap. In HQMC’s plan DL solutions will be network- 
delivered where possible; exploiting the power of the Internet in order to distribute and 
track computer-based interactive multimedia instruction (IMI). Realizing that a robust 
inffastracture is required to support a network-based learning environment, interim 
hybrid solutions are already in the works. The roadmap further outlines solutions that 
include a combination of the Internet, CD-ROM technology and traditional paper-based 
DL courses, until the proposed enterprise network can adequately support network 
delivery. The plan also describes how this network-based learning will be additionally 
supported with video teleconferencing and teletraining capabilities to leverage existing 
service and government video networks. 

Finally, for the long term the HQMC DL Roadmap identifies the development of 
artificial intelligence, intelligence agents, intelligent tutoring systems, performance 
support and embedded training technologies that will be encouraged during new system 
acquisitions to reduce the requirement for equipment specific training. 

C. USMC DISTANCE LEARNING STRATEGIC VISION 

Existing and emerging information technologies provide an excellent opportunity 
to deliver flexible and adaptable training and education solutions. The goal of USMC 
training and education programs is to encompass a “cradle to grave” approach that begins 
with initial entry and specialized skills training and continues through career-long 
professional development (HQMC T&E DL ROADMAP 1999). 

The Marine Corps developed the Training and Education Modernization 
Initiative (TEMI) as a substantial part of its overall educational strategy to meet new 
challenges with increased flexibility. The objective of this initiative is to maximize the 
Corps’ limited training and education resources by restructuring current institutional 
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training, improving existing training design/development and training management 
processes, while simultaneously introducing technology improvements into classrooms 
that capitalize on modem DL technologies. Professional Military Education (PME) 
programs will also benefit from this initiative as current distance education courses are 
enhanced through the application of advanced/emerging information technologies. 

D. OPPORTUNITIES AND CHALLENGES 

The question naturally arises as to why DL and training technologies should be 
embraced? An increased emphasis on the use of DL technologies will greatly enhance 
training and education opportunities for all Marines for a number of reasons. First, there 
is a momentum across industry, academia and the other services to exploit modem 
technology for the purpose of improving instruction and increasing opportunities for 
access. Recent advances in technology have made DL an increasingly viable option to 
deliver courses of instruction on demand, with minimal regard to time and distance. 
Secondly, the Executive Branch has specifically tasked the services as well as all other 
Federal agencies to increase the use of DL where possible to reduce the cost of 
institutional training and education programs (EO 13111, 1999). Finally, DL has the 
potential to “level the playing field” for reserve forces by providing them with increased 
opportunities for training and education. Expanded use of DL will help reduce resident 
training time and training and education costs, while increasing educational opportunities 
for all Marines (Tyler, 1998). 

The military and civilian industry can no longer afford to conduct training and 
education in the same outmoded, predictable fashion. The challenge today is to meet 
increased training and education demands in spite of reductions in funding, manpower 
and training infirastmcture. While resources are decreasing, requirements for training and 
education are constantly increasing. Changes in demographics, procurement practices 
and organizational structure have resulted in a younger force, new equipment, changing 
missions and MOS mergers; all of which impact training and education requirements. 
Studies in both civilian academic and military training environments assert that DL offers 
a means to meet these emerging and fluctuating demands (Moore & Kearsky, 1996; 
Freeman, 1999). 

Distance Learning applications can reduce the cost of traditional training and 
education while still meeting traditional training and education needs (OUSDP&R, 
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1999). A study conducted by the Logistics Management Institute (Belcher, 1999) states 
that use of network and CD-ROM distribution in DL can significantly reduce the 
production and distribution costs associated with conventional paper-based courses. 
Phelps, Wells, Ashworth and Hahn, (1992) convincingly demonstrate how offering a 
course via DL can also reduce or eliminate travel and per diem costs. An analysis of two 
identical study groups, one in residence and the other using Computer-mediated 
Communications (CMC) technology from the U.S. Army’s Engineer Officer Advanced 
Course, resulted in cost savings in excess of $600K based on five iterations of the course. 
Course topics included technical engineering skills as well as Army tactics, leadership 
and briefing and presentation skills. The Battle Staff Noncommissioned Officers’ Course 
(BSNCOC) at Fort Bliss, Texas, estimates an average cost savings of more than $30K per 
student for its six-week course (Belcher, 1999). Adopting DL practices can also provide 
widespread access to training and education resources. For example, it can increase 
student throughput as illustrated by the Air Force Institute of Technology (AFIT), which 
increased student throughput from 300 to 3000 for its Acquisition Planning and Analysis 
course by converting it to a DL format (IDEA, 1999). The Naval Postgraduate School is 
pursuing a similar course of action for several of its graduate education programs 
(Hazard, 1999). Finally, the Army National Guard is developing comprehensive DL 
programs to increase training opportunities for over 36,000 Guardsman across the United 
States (Metzko, 1996). 

Advanced technologies are currently being explored to develop and deliver 
learning products just in time, when and where they are most needed. In the future. 
Marines can expect to use a Navy/Marine Corps Intranet (N/MCI), the Internet, learning 
resource centers (LRCs), interactive multimedia instruction (IMI), video teletraining 
(VTT) and embedded training (ET) to master new skills. Although Distance Learning 
technologies have the potential to yield savings, they can be costly to initiate, requiring a 
substantial up-front investment (Tyler, 1999). The cost of implementing technology that 
fails to live up to expectations, highlights the importance of a coordinated approach to 
research, development, acquisition and life cycle management. Acquisition projects, such 
as the USMC’s DL Program are therefore consulting training design specialists early in 
the requirements definition phase to reduce the cost of training and align it with the 
existing Marine Corps training standards (MCSC DL LCCE, 1998). The continued 
partnership between the acquisition force, manpower specialists and tr ainin g has the 
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potential to achieve significant life cycle cost efficiencies for major Marine Corps 
equipment procurements in the future. 

Many of these technologies are still under development and require fiirther study 
to ensure that each will be a cost-effective solution to growing training and education 
requirements. Likewise, the partnership between acquisition, manpower and training 
remains in its infancy and must be cultivated to fruition. Moreover, development and 
maintenance of long-term partnerships with other services remain crucial to benefit from 
their technological advances, while meeting Marine Corps specific requirements and 
defraying overall costs. 

E. MODERNIZATION TENANTS 

Training and education is an investment in operational readiness. Restructuring 
existing programs must focus on enhancing operational readiness by making training and 
education better and more efficient. The basic tenets of the modernization initiative are 
as follows (HQMC T&E DL ROADMAP, 1999): 

• Training and education are a core responsibility of the Service 

• Operational readiness is the primary consideration for implementing training 
and education programs 

• Marine Corps training and education will be standards-based 

• A Marines’ educational experience is part of a career-long learning continuum 
supporting the operational needs of the total force 

• Technology will be leveraged to improve the effectiveness and efficiency of 
Marine Corps training and education 

• DL will increasingly be used to meet future Marine Corps training and 
education requirements 

• The Marine Corps will leverage other DoD and governmental agency DL and 
instructional technology efforts 

F. THE MARINE CORPS DISTANCE LEARNING PROGRAM 

1. Overview 

The revised Marine Corps DL program expands on the paper-based system of the 
Marine Corps Institute (MCI) by developing integrated training and education programs 
for initial skill and skill progression training that lead to Military Occupational Specialty 
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(MOS) qualification, and also enhance current distance education PME programs. The 
DL program is a funded Total Force program supporting both the active Marine Corps 
and the Marine Corps Reserve. Implementing comprehensive DL programs is a complex 
undertaking and involves developing innovative solutions in four major technology 
domains: instructional content, delivery infrastructure, instructional management systems 
for delivering courses and managing students in a distributed environment and instmctor 
training and support (HQMC T&E DL ROADMAP, 1999). The DL program will offer 
solutions in each technology domain and will be implemented according to a two-phase 
schedule - a pilot phase (FY97-99) and a program expansion phase (FYOO-05). The pilot 
effort was initiated to study infrastructure and process and resource requirements, to 
successfully establish a viable DL program. Lessons learned from the pilot effort are 
being used to shape the DL program so that limited resources can be effectively focused 
on establishing a flexible Corps-wide DL capability in the future. A major part of 
implementing the USMC DL program will center on the establishment and activation of 
the Marine Corps Learning Network or MarineNet infrastructure. As of this writing, it is 
envisioned that MarineNet will function as a subcomponent of the larger N/MCI that is 
currently being planned (CNO, 2000; Whitbeck, 2000). For the purposes of this paper, 
the use of the terms N/MCI and MarineNet will be synonymous. MarineNet is a wide- 
ranging initiative that provides the supporting infrastructure and access points to enable 
the delivery of high quality training and education to all Marines, regardless of location. 
MarineNet is essentially a Marine Corps-wide distributed Intranet supported by 
compressed video technologies that will enable Marines and civilian DoD personnel to 
learn via the appropriate media when and where learning is most needed and required 
(HQMC T&E DL ROADMAP, 1999). MarineNet is composed of electronic interactive 
DL courseware, hardware, software and network components necessary to distribute 
electronic instmction over Marine Corps wide area, metropolitan area and local area 
networks. 

The N/MCl will use the planned Defense Information Infrastmcture (DII) 
architecture for wide area connectivity across the Marine Corps enterprise, and the 
planned Base Telecommunications Infrastructure (BTI) upgrades for metropolitan eirea 
and local area network connectivity aboard bases and stations (HQMC T&E ROADMAP, 
1999). The R-net will provide network connectivity to the 194 reserve sites across the 
country (Metzko, 1996). Open system architecture will be used to ensure interoperability 
between Marine Corps and other service DL and communications systems and platforms. 
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2. Organizational Roles to Support USMC DL 

The DL program and MarineNet will be supported by the following 
organizational components: the Distance Learning Center (DLC), Functional Learning 
Centers (FLCs) and Area Learning Centers (ALCs). These components conform to a 
three-tier approach for DL. Each center corresponds to an echelon within the Marine 
Corps and has specific roles and responsibilities to that organizational level. As 
mentioned previously, the components of the functional structure are physically 
interconnected via the BTI. The BTI will provide an upgraded network infrastructure 
aboard all bases and stations as well as telecommunications “pipes” between Marine 
locations via the Defense Information Systems Network (DISN) (HQMC T&E DL 
ROADMAP, 1999). 

The MCI designated as the DLC, is the Corps-level organization that will provide 
Marine Corps-wide standardization, certification and quality control for all DL in the 
future. The DLC will provide a consolidated Corps-wide on-line catalog of DL products 
to include a VTT broadcast listing, accessible through the Internet. The DLC will also 
manage master DL manpower data, ensuring complete and accurate information on 
enrollments, completions and qualifications are passed on to the Marine Corps Total 
Force System (MCTFS). 

The formal schools currently in existence will serve as FLCs and functional area 
proponents, managing all electronic DL courses related to their specific area of expertise. 
Multimedia course development and maintenance will occur at Regional Development 
Centers (RDCs). RDCs are supporting organizations that will also manage contractor- 
developed DL products, ensuring that they conform to established Marine Corps’ 
standards and protocols. RDCs will be xmder the operational control of the DLC and 
under the administrative control of the local base establishment. Electronic course 
distribution will occur from FLC servers, while paper-based distribution functions will be 
retained by the MCI. 

The RDCs will provide centrally managed and dedicated course development 
teams at selected Marine Corps Schools (i.e., FLCs). The two FLCs initially selected to 
host RDCs are the Marine Corps Communications and Electronics School, located at 29 
Palms, California, and the Marine Combat Service Support Schools, Camp Lejeune, 
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North Carolina. The RDCs will support the FLCs in their respective geographic locations 
(i.e., East CoastAVest Coast). Each RDC will be responsible for developing technology- 
based courseware and managing contractor developed content for its client FLCs. The 
FLCs will provide actual course content and subject matter expertise. The course 
development section of the MCI will provide content development support to smaller 
formal schools/detachments, as well as assist with overflow course development from the 
two RDCs. Establishing an RDC for PME course development at the Marine Corps 
University is scheduled to begin in FY 00. 

The ALC is the primary metropolitan area network for delivering DL courseware 
to Marines in a given geographic area. The ALC will be comprised of a Training and 
Education Point of Presence (TEPOP) server with one or more interconnected Learning 
Resource Centers (LRCs) and VTT Centers. The number of LRCs and VTT Centers 
within the ALC geographic region will depend upon the size of the region and the 
population size supported by the base, station or site. The ALC will provide local DL 
account management, storage and distribution of electronic courseware to any authorized 
user connected to the ALC network. 

The primary access to MarineNet will occur at the local level ALCs. The ALC 
TEPOP server suite is designed to provide local (regional) instructional material storage, 
distribution and security services. One TEPOP is required per ALC (region) and will 
service many “tenant” FLCs and LRCs. The storage resource will provide the capacity 
for all electronic training material to be accessed through workstations aboard the base, as 
well as the necessary management tools to monitor student progress, monitor network 
utilization, determine courseware availability and suitability and maintain statistical 
information. 

The LRC is designated as the primary location to access DL courseware for those 
Marines who do not have access to computer workstations. The LRC is a client-server 
Local Area Network (LAN) system connected to the base network backbone and accesses 
courseware stored on the regional TEPOP server suite. Each LRC can accommodate 
approximately 25 to 40 simultaneous users. Approximately 94 LRCs will be procured as 
part of the DL program in addition to the workstations already connected to the Marine 
Corps enterprise network (HQMC T&E DL ROADMAP, 1999). 

Where established, VTT Centers will provide the capability to conduct DL using 
the latest Video Teleconferencing (VTC) technologies. The centers will have a two-way 
video and two-way audio (2V/2A) capability and the ability to accommodate 15 to 20 
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simultaneous users. The system will conform to all DoD standards and guidelines and 
will be capable of multi-point conferencing with all VTT centers DoD-wide (Cabrera, 
1999). The MarineNet VTT systems will leverage the existing Marine Corps Satellite 
Education Network (MCSEN), and the Navy’s CNET Electronic Schoolhouse Network 
(CESN) capabilities. The DL program will field 25 additional VTT sites that will be 
interconnected with the MCSEN and CESN systems at both active and reserve locations. 

Plans also call for deployable LRCs that will provide operational units with the 
capability to access DL resources while deployed abroad (HQMC T&E DL ROADMAP, 
1999). These small self-contained, raggedized client-server networks will emulate the 
capability of the fixed site LRC. Further, they will have the capability to connect to 
shipboard or external Transmission Control Protocol/Intemet Protocol (TCP/IP) 
networks. The system is composed of a server and ten client workstations. Courseware 
will be uploaded onto the deployable server prior to deployment and updated through a 
“reach back” capability to the host unit TEPOP where adequate long haul 
cormmmications links exist. 

The Marine Corps DL program relies upon the implementation of the BTI 
upgrade. Delays in BTI network implementation will thus defer the fielding of 
MarineNet (Tyler, 1998). Program sponsors are currently working closely with the 
Marine Corps Systems Command and the bases/stations to coordinate BTI 
implementation with MarineNet fielding (Tyler, 1998). Successful integration with the 
R-Net is also critical to the success of the DL program for the Reserve component (Tyler, 
1998). R-net integration issues will be thoroughly studied during the pilot phase of the 
DL initiative to ensure adequate interoperability before program expansion in FY 00. 

3. DL Content Development and Management Issues 

An integral part of the Marine Corps TEMI also involves a comprehensive review 
of all formal training tacks. The Training and Education Division will lead the effort to 
conduct detailed training reviews for 100 percent of all institutional training courses over 
the next six years (HQMC DL ROADMAP, 1999). Training reviews will commence 
with those occupational fields determined to have the greatest potential savings in terms 
of MAT and transient times. Further, those courses with insufficient capacity to meet 
throughput requirements will be the first candidates for review. The objectives of the 
training review process are: 
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(1) Design a comprehensive MOS training concept. 

(2) Identify essential job performance competencies and supporting tasks for 
MOS qualification. 

(3) Design a training progression model that meets career track requirements for 
each MOS. 

(4) Redesign courses as appropriate to incorporate both resident and DL to 
achieve MOS qualification within established resource guidelines. 

Once the curriculum reviews are completed, FLCs will design and develop both 
resident instruction and DL products to support the new training progression models and 
curricula. Course development efforts at the formal schools will be supported by the 
RDC and MCI development teams, or through commercial outsourcing. The 
implementation of revised integrated curricula will vary depending upon the complexity 
and subject matter involved; however, the time window for this process is to begin 
delivery of new training and education solutions within six to nine months of 
commencing the review process. 

Another component of the DL program is an enterprise-wide management system 
to coordinate courses and students in a distributed environment. The system must be 
flexible to adequately support a highly mobile population of active duty and reserve 
Marines, to include civil servants for on-line registration, tracking and assessment. 
Further, the system must be able to accommodate not only training events, but PME and 
voluntary education opportimities as well. The system of the future will provide an 
alternative registration and assessment capability for traditional DL courses until they are 
eventually phased out, and serve as a secure gateway for new on-line DL courseware and 
materials. The system is planned to be accessible through both MarineNet and the 
conventional Internet. 

From a human resource perspective, a further consideration is that DL is a 
relatively new operating environment for Marine Corps instructors and curriculum 
developers. Exposing these instructors, training developers and senior leaders to 
emerging DL instructional technologies will greatly assist them in overcoming the normal 
apprehension associated with applying new methods and training concepts. Accordingly, 
instructor development and support are critical factors to the overall success of the DL 
program. To help facilitate this transition, the DLC and the Instructional Management 
Schools located on both coasts will develop instructor-training programs and provide 
technical support to ease the expansion of DL capabilities across all Marine Corps formal 
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schools. Training programs will include information on the instructional design process, 
implementation of DL instructional technologies and project management. Additionally, 
the DLC will provide a help desk to support DL instructors and curriculum developers at 
the RDCs and FLCs. 

This review process is a significant effort. It will require time and dedicated 
effort on the part of all relevant parties to develop pertinent and cost effective training 
solutions for each of the MOSs considered. This situation does however, present the 
Marine Corps with an opportunity to enhance training and education through a 
restructuring effort that will yield even greater dividends in the future. 

G. CHAPTER SUMMARY 

The objective of the TEMI and more specifically the implementation of DL, is to 
provide better and more effective training in an environment of increasingly fewer 
resources. Initiating modem instmctional technologies to both deliver DL and 
simultaneously enhance our resident school training, presents significant but not 
insurmountable challenges. Meeting the established goals of reducing MAT, overall 
training time and training structure, while providing better and more efficiently delivered 
instmction to Marines is only a part of the challenge. Shifting to a learner-centric vice 
instructor-centric approach to learning, while ensuring computer literacy, making limited 
time available for Marines to train via DL techniques, and also overcoming institutional 
bias and resistance to change, present challenges of equal if not greater magnitude. 

Distance Learning, although not a panacea for all training and education issues, is 
an established and proven method for delivering training and education without 
sacrificing quality. MCI has been providing paper-based DL to Marines for more than 75 
years with great success. Modem technology provides an opportunity to significantly 
improve the quality of the learning experience. Further, such technology will enable the 
Marine Corps to successfully achieve its goal of providing more Marines a higher quality 
of training and education while reducing the required resources. 
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III. ELEMENTS OF ROI FOR DISTANCE LEARNING 


A. INTRODUCTION 

Literature about the cost effectiveness of distance education tends to focus on 
costs of specific media implementations (Rumble 1989; Solomin and Holden 1988); 
pragmatic discussions of models and costing issues (Markowitz 1987; Rumble 1988); and 
comparisons between traditional resident instmction and distance education (Hahn, 
Ashworth, Phelps, Wells, Richards & Daveline 1991; Laidlaw and Layard 1974; Muta 
1985; Rule, DeWulf & Stowitschek 1988; Wagner 1977). The literature suggests that 
distance education can be less expensive than resident instruction, depending upon 
student enrollment and the fixed costs of course development and delivery. Furthermore, 
the literature compares the training effectiveness of distance education to face-to-face 
resident instruction, and suggests that distance courses delivered with various media 
compare favorably with resident instmction. The conclusion that such effectiveness is 
equal among these two methods is not generally accepted, however, due to disagreement 
over such issues as the value of human interaction in training and education (Beare 1989; 
Jevons 1982; Misanchiik 1982; Whittington 1987). 

The major challenge of distance education is increasing access while controlling 
the cost of delivery. Meeting this challenge while maintaining effective instruction is 
cmcial to achieving the return on investment necessary to insure the viability of distance 
learning programs. The imperative of access and cost effectiveness requires the selection 
of the least expensive alternative that meets the course objectives and reaches the 
intended audience. All things being equal, the more access and less expensive the 
method, the better for all concerned. Courseware designers and managers are expected to 
be good stewards of resources entmsted to them seeking efficiency in their designs. 
Executive Order 13111 recognized an obligation to increase access while controlling 
costs. 

It would appear that for all the emphasis placed on minimizing training expenses, 
a thorough cost analysis of training alternatives is not an easy thing to do, particularly 
where DL technology is concerned. Variables can be many and ill defined. The 
remainder of this chapter provides insight to those elements critical in obtaining a 
favorable ROI for DL. Issues include a discussion of cost benefit analysis (CBA) versus 


19 



cost effectiveness analysis (CEA), a review of cost-benefit techniques, factors of cost 
analysis in training and development, and a look at measures of effectiveness commonly 
used in determining ROI for DL. 

B. ISSUES AFFECTING ROI FOR DL 


1. Overview 

If one can deliver instruction to many students at their home stations 
economically, there surely must be cost savings over bringing those same students to 
centralized training. Russell (1999) argued that DL can reduce training travel and per 
diem costs compared to schoolhouse-based instruction, and that the training results in no 
significant difference in training effectiveness. This reasoning, although supported by 
over 300 studies, is still considered highly subjective and is far from conclusive. 
Discussion comparing the effectiveness of these training methods is a very complex 
issue, and is beyond the scope of this study. To avoid confusion in the remainder of this 
paper, the context of effectiveness will refer to cost analysis only. 

Distance Learning involves more than technology. It is a combination of people, 
process and technology that creates effective and efficient DL programs (Kidwell 1998). 
There is an rmderlying study of pedagogy that goes into determining what DL option, if 
any, makes sense for a particular course. Most DoD and service DL strategies recognize 
this requirement that training be student-centered, and not technology-centered. 
Questions remain, how should the military services decide whether to invest in DL 
programs? How should they determine in which technologies to invest? How should the 
DoD measure the value of those investments? What costs and benefits are relevant? The 
following review may help the reader understand the factors that contribute to arriving at 
a decision as to the applicability of DL solutions in a learning environment. 

2. Synchronous versus Asynchronous Delivery 

Distance education delivery systems are categorized as either synchronous or 
asynchronous. Synchronous delivery requires instructors and students to participate at 
the same time, while asynchronous delivery allows participation at differing times. The 
main advantage of synchronous delivery is the provision for live interaction and the 
possibility of more natural group processes. The disadvantage is the requirement to 
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adhere to a specific time frame that may not be convenient for all participants, especially 
those in other time zones (Steiner, 1995). Asynchronous delivery systems are 
characterized by the separation of the instructor and student in time. Asynchronous 
systems allow anytime-anywhere learning, but are more limited in student and instructor 
interaction. 

As cited in Christensen, et al. (1998), Alan Chute points out that synchronous 
events are desirable in distributed learning programs in order to provide student-to- 
student interaction for peer learning, and student-to-instnictor interaction for mentored 
learning. Additionally, synchronous events provide a framework of calibration and 
expectations to keep students on a scheduled track. Even programs that depend primarily 
on asynchronous learning benefit from periodic synchronous events. Christensen, et al. 
(1998) fiirther discusses how synchronous learning activities leverage the great depth of 
expertise and qualities of the traditional classroom, while expanding the physical reach to 
geographically separated learners. 

The key here is to be able in some way to quantify the benefits of a synchronous 
training environment versus an asynchronous one in order to determine which is the more 
appropriate and valuable for a particular set of circumstances. The following subsections 
list several additional factors that may contribute to potential returns on DL, which 
should be evaluated when assessing the worth of DL programs. These factors may 
impact either positively, negatively, or not at all, but should be considered nonetheless. 

3. Benefits Of Potential Returns from DL 

TIME SAVINGS. With DL, the amount of time students spend in training can be 
greatly reduced. The time previously spent in travel also becomes available for work or 
training. This time “compression” factor is almost always a positive result of course 
conversion to a DL format (Belcher, 1999). A well-designed DL program requires less 
student time to achieve learning objectives than does traditional classroom instruction. 
(An additional value-added result of this process is that the course Program Of Instruction 
(POI) is often updated or modified during the conversion process, resulting in a shorter 
overall course.) Belcher (1999) states that asynchronous learning allows students to skip 
ahead if they already know some of the material. He further suggests that because 
students can progress through these lessons at their own pace, comprehension may be 
enhanced. Some technologies allow students to return to material on the job as refresher 
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training. Studies conducted by the Institute for Defense Analysis suggest that this 
compression is typically about 30 percent for most forms of DL and is occasionally as 
high as 60-70 percent (Metzko, 1996; Howard, 1997). For the purposes of this study, the 
more conservative value of 30 percent will be used. 

UNIT READINESS. Trainers argue that the ultimate objective of any military 
training or education course is to bolster readiness (and subsequently, performance) of 
military units. The Army’s DL plan says as much (TADLP, 1999). DL technology 
enables trainers to claim these readiness benefits: 

• Because some training can be extracted from schoolhouse-based instruction 
and done in the operational units, recruits can spend less time at initial 
schooling and be assigned to operational units faster. They would then 
receive the additional training later as required. 

• Units can get larger portions of their organizations trained on critical subjects. 
Since students take less time to learn some distance learning courses, they 
spend less time away from imit duties and training than if they took traditional 
classroom instruction. Personnel are also available in the event of an 
emergency. The result is improved overall readiness. 

• The ability to take training with them (e.g. on a laptop computer) enables units 
to keep personnel even during major field exercises and deployments, 
enabling the personnel to continue training. 

ANYTIME, ANYPLACE TRAINING. As mentioned before, one of the ways for 
DL to be student-centered is for the student to decide how learning takes place. The most 
efficient DL programs will give the student volitional control over when and where to 
train. With minimal support equipment, some distance learning technologies enable 
trainees to access courseware at the time and place of their choosing. Computer-based 
Instruction (CBI) and CD ROM-based instruction require only a computer of modest 
capability. Internet access naturally extends that accessibility to Web-based instruction. 

ENHANCED COHESION AND MORALE. With DL, trainees spend more time 
at home stations, near fiiends and family, which enhances morale (Belcher, 1999). This 
is particularly true for Navy shipboard personnel who spend much of their time afloat. 
Distributed courses thus allow for more student and/or coiuse flexibility over 
conventional schoolhouse training. The advantage is tempered somewhat from the unit’s 
perspective, however, by the need to provide personnel duty time to complete required 
training. 
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REDUCED TRAVEL COSTS. The most widely reported monetary benefit of DL 
is that reduced in-residence classroom instruction minimizes the requirement for trainees 
to travel to schoolhouses. Depending upon the size of the training audience and the 
length of the course, this can reduce travel and per diem costs. Travel and transportation 
savings can be compounded when training units use linked simulations or networked 
simulators. The cost of transporting entire units and their equipment to a central training 
area can be avoided (Belcher, 1999). 

D>JCREASED EFFICIENCY. Although initial costs for many forms of technology 
and course conversion are high, they can be offset by higher course enrollments over 
time. The increased number of people trained therefore, can thus recoup the start-up 
costs more quickly. An increasing portion of future training infrastructure will be virtual. 
Requirements for traditional schoolhouses and training facilities will diminish, reducing 
overall infrastructure costs. Finally, the groundswell for Distributed Learning is driving 
the market toward lower-cost solutions. As competition between providers heats up, 
technology improves and hardware and software costs will decline. 

4. Costs of Potential Returns from DL 

The costs associated with particular DL programs/technologies entail equal 
consideration and include some of the following factors: 

COURSE CONVERSION COSTS. Costs that should be considered when 
assessing possible conversions are hardware, software and transmission conversions, as 
well as time needed to train faculty and staff (Martin & Bramble, 1996). On the plus 
side, technology-driven items such as hardware and software items are decreasing, but 
facilities and communications infrastructure costs may apply for selected new courses. 
Doing the work of conversion in-house can minimize course conversion costs. The two 
principal considerations to lower course conversion costs are to have the course up to Hatp 
with good presentation material, and to have well-developed standards and guidelines for 
the courses (Payne, 1996). 

SUPPORT COSTS. For asynchronous Web-based training, building and 
maintaining a Web site requires planning for system maintenance and availability. If 
students are located around the globe, the site should be hosted at a location that provides 
24 hours per day, 7 days per week support. For synchronous training, technical support 
personnel must be on-hand during sessions. Instructors and students may be working 


23 




within narrow schedules, resulting in a domino effect if lessons are delayed with no 
rebroadcast (Robinson, 1998). Costs are also associated with “training the trainer.” DL 
instructors must be more organized than classroom instructors. DL instructors require in- 
depth knowledge of course material and of the training technology. Some forms of DL, 
like VTT can be relatively expensive to operate and maintain, but these costs often are 
overshadowed by the travel cost savings and the ability to train larger numbers of 
students simultaneously (Belcher, 1999). 

STUDENT EQUIPMENT. As DL becomes computer and Internet based. 
Internet-capable student computers become an absolute requirement to access instruction. 
Unfortunately, not all students own or have access to these machines (Robinson, 1998). 
Training planners should account for the cost of equipment required by the students, or 
alternatively, they must ensure that such access is afforded to students. As the equipment 
requirements change over time, DL planners must determine how the school or course 
will keep up with those changes. 

5. Neutral Factors that May Influence Returns from DL 

Planners should ftuther be cognizant of various considerations that fall neither in 
the benefits nor costs categories. Many of these issues surface as causes for concern for 
organizations currently grappling with DL implementation issues. 

ACCOUNTING FOR INVESTMENTS. Distance Learning conversions are often 
the products of various initiatives at various levels and funding sources. Organizations 
making the investment in learning may not necessarily be the same organizations reaping 
the benefits (ODUSDR, 1999). Hardware and software supplied as government- 
furnished equipment (GFE) to DL programs and developments that result fi-om 
multiservice or govemment/industry collaboration further complicate the investment 
chart of accoimts. 

MATCHING NEEDS AND TECHNOLOGY. Trainers must be careful not to let 
technology drive strategy. There is a risk that plans will be made and technologies 
selected before the requisite analysis is performed. A careful match of the training needs 
and the technology available should result in cost savings. According to Sherry (1996), 
“too often, instructional designers and curriculum developers have become enamored 
with the latest technologies without addressing the underlying issues of learner 
characteristics and needs; the influence of media upon the instructional process; equity of 




access to interactive delivery systems; and the new roles of teacher, site facilitator and 
student in the distance learning process.” Student characteristics also must be considered 
when designing a course (i.e., independent work without supervision, number of low 
cognitive tasks, etc.). The student, not the technology, should be the focus of the design 
(Belcher, 1999). 

COMPLICATED RESOURCE SCHEDULING. Particularly for VTT courses, 
both transmission and downlink site availability must be coordinated for each class. This 
task can be complicated for synchronous training courses transmitted to remote locations 
over several time zones. Such off-hours staffing and operations add to operations and 
support costs (Belcher, 1999). 

SCHOOLHOUSE IMPACT. Traditional schools must digest a new philosophy. 
Training must now follow a decentralized, student-centered model. Belcher (1999) states 
that training should be designed for how students learn, not for how schools teach. To 
him, flexibility is the key. In a student-centered model, the school must be flexible 
enough to respond to the needs of the individual learner. Since DL courses yield higher 
student throughputs, they can create backlogs or bottlenecks for other related training 
(Metzko, 1996). Schools may make portions of a course available to students via DL 
technology, but they may not produce more trained personnel because a subsequent phase 
can only accommodate a few trainees (Belcher, 1999). As such, a more holistic view of 
the training and education process must be considered when implementing DL solutions. 
Furthermore, resource managers need to consider new ways to budget for schools. In the 
past, military service schools were allocated resources based on student throughput. In 
the DL environment, this approach may no longer be an appropriate measure. School 
budgets should now focus on the number and type of courses the school must convert and 
execute (Belcher, 1999). Finally, instructor training/certification requirements must be 
redefined. The new teaching environment dictates that instmctors be certified to teach 
using current DL technology. Critical issues here center on instructor competency with 
hardware/software components, as well as flexibility in established teaching procedures 
and curricula. 

Belcher’s (1999) assessment is that drawing conclusions about ROI for DoD as an 
enterprise is complicated. Many initiatives have been undertaken across the services. 
Naturally, there has been some redundancy and duplication among them. It is reasonable 
to assume that eliminating duplication across the four services could equally reduce up¬ 
front investment costs (Metzko, 1996). The results, however, are not purely additive; 
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determining just what constitutes duplication is subject to interpretation. Case-by-case 
analyses are required to understand the synergies, opportunities and risks associated with 
determining ROI on DL. 


C. COST-BENEFIT VS. COST-EFFECTIVENESS ANALYSIS 

Cost analysis can help educational managers see the various options and trade¬ 
offs available to them and assess their merits and feasibility. It can reveal the possible 
advantages of redeploying limited resources between different levels and types of 
education, among different categories of inputs, and across different geographic areas. 
Cost analysis can uncover internal waste and recommend possible ways to eliminate it. It 
can also suggest ways to enhance the external productivity of the educational process and 
the benefits accruing to individuals from investments in technology and new processes. 

Cost analysis must be combined with pedagogical analysis to assess the outcomes 
of any educational process. The key to judging the efficacy of an educational process lies 
in comparing the costs invested in it and the learning results attained. There is a 
difference between cost-benefit and cost-effectiveness. As Reynolds and Ivrinski (1996) 
state, cost-benefit analysis is an attempt to weigh the costs of training against the 
outcomes achieved. Cost-effectiveness focuses on comparing the costs of two or more 
training alternatives, or examines ways to reduce a program’s costs, while cost-avoidance 
involves simply cutting expenses to a set level. This paper is chiefly concerned with 
cost-benefit analysis. According to Gramlich (1998), the fundamental principle of cost- 
benefit analysis is that when choosing among programs, the best course of action is to 
choose the one that maximizes net benefits. To accomplish this, add up all the gains from 
a policy alternative, subtract all the losses and choose the option that maximizes the total 
benefits. 

Since life doesn’t always present simple alternatives, however, there are some 
additional guidelines to use in applying this fundamental rale. According to Gramlich 
(1998), one should not compute cost-benefit ratios. He feels that the right measure of the 
gains or losses to society is program net benefits, and the cost-benefit ratio can be a 
misleading guide. His second rale of thumb is that a cost-effectiveness test can be used 
to get around missing information when the investigator is comparing two or more ways 
of accomplishing an end. A third consideration is that when projects are packaged 
together, the net benefits test should be applied to the whole package (Gramlich, 1998). 
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Cost-benefit analysis is a framework for comparing the pros (benefits) and cons (costs) of 
project choices. Benefits and costs should be quantified whenever they can be and 
(obviously) not when they cannot be; but whether quantified or not, they should never be 
ignored (Coombs, et al., 1987). 

D. COST-BENEFIT TECHNIQUES 


According to Reynolds and Iwinski (1996), there are four considerations 
ordinarily used to analyze cost-benefits: 

BENEFITS. Quantifying benefits is a challenging task, because the outcomes of 
training are often intangible or diffieult to translate directly into dollar amounts. Factors 
such as increased student throughput, reduced travel and per diem expenses and savings 
in realia expenses (i.e., the cost of using real things such as ships or aircraft) come most 
readily to mind. Job performance or organizational results, although difficult to quantify, 
oftentimes more accurately justify a new training program, and can be expressed in 
financial terms as well. Most technical training is much easier to quantify than 
management training because it is directly linked to processes and products with known 
or measurable value. 

LIFE CYCLE. One of the basic elements of CBA is life cycle cost. DoD 
Instruction 5000.2 (DoD, 1991) defines life cycle cost as follows: 


Life-cycle cost reflects the cumulative costs of developing, procuring, 
operating, and supporting a system. They are often estimated separately by 
budget account (i.e., research, development, test, and evaluation..., procurement, 
and operations and maintenance). It is imperative to identify life-cycle costs, non¬ 
monetary as well as monetary, associated with each alternative being considered. 

Costs accrue over the life of a system. TRADOC divides this life into five distinct but 
sometimes overlapping phases (Department of the Army, 1985): 

• Conceptual (exploratory development): Solicitation, evaluation, and 
exploration of alternative concepts. 

• Demonstration and Validation (advanced development): Protot)q)es 
are produced to support demonstration. 

• Full Scale Development (engineering): Prototypes are produced to 
support operational test and evaluation. 
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• Production and Deployment 

• Operation and Support 

Often you can only determine true cost savings by considering the entire lifetime 
of the training project. The life cycle model is shown in Figure 3.1. The life cycle 
method’s strong point is that you can evaluate the total costs of each of the planned 
program’s phases to determine whether it will result in a net cost savings. Figure 3.1 
illustrates the expenditure profile for a typical training project. Expenditure corresponds 
to new implementations. The figures show that some money must be spent before 
anything occurs. These costs are labeled as research and development even if no research 
is involved. The next phase is called startup and marks a sharp rise in money spent on.the 
training technology since it has now come of age. The operational period, sometimes 
referred to as the “steady state,” is the main period during which the program is active 
(Reynolds & Iwinski, 1996). This time frame is much longer for some technologies than 
for others. Often a program that performs its fimction well and remains current, is 
delivered by an obsolescent technology for years. Finally, as the program ages, it is 
gradually phased out or eliminated during the decline phase. This process is normally 
accompanied by a simultaneous transition to a new program. 

Training Costs 


STEADY STATE 



R&D Start Up Operational Period Decline & 

Transition 


Figure 3.1 Technology Life Cycle from Reynolds & Iwinski (1996) 









PRODUCTIVITY. This consideration compares both a program’s efficiency and 
its effectiveness. It may be used to demonstrate that a project reduces training costs or 
increases training. A productivity analysis can determine when to switch fi-om one 
training approach to another, or when further application of a particular approach will no 
longer produce cost-effective improvements. 

RESOURCE REQUIREMENTS. This component is the simplest and most direct 
way to compare the costs of two or more different training approaches at a given time 
(Reynolds & Iwinski, 1996). One merely determines the costs in four major categories 
for each Instructional Systems Development (ISD) phase (analysis, design, development, 
implementation and evaluation) of the particular training project. Make use of the cost 
categories equipment, facilities, materials and personnel. The result of the analysis will 
include the total costs for each phase and category. Resource requirements work well in 
comparing a potential new method with an existing one, assuming that both applications 
will be equally effective. To compare two possible approaches, one simply adds the total 
costs of each for all phases and resource categories. Resource requirements are limited to 
training costs, however, and should not be used to compare effectiveness. To properly 
conduct a worthwhile cost analysis, one must understand the various factors that 
comprise cost estimation. The following section will review some simple fundamentals 
of cost analysis to include basic economic principles and intangible influences. 

E. COST ANALYSIS IN TRAINING DEVELOPMENT AND ROI 

1. Cost Estimation 

Cost estimation should take into account several economic factors. The following 
is distilled from a description of key factors in Adams and Rayhawk (1987). 

• Opportunity Costs versus Accounting Cost: Accoimting Cost is the 
cost “on the books.” Opportunity cost is a hypothetical value of a 
resource in it “best alternative use.” 

• Sunk Costs: Costs that have already been incurred and that cannot be 
recouped. An example is the cost of R&D spent on various forms of 
technology. 
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• Fixed and Variable Costs: Fixed costs are not affected by how much 
training occurs. An example would be the cost of classroom space. 
Variable costs vary with the amoimt of training. An example would be 
the cost of instructors, whose number would vary with the student 
load. 

• Time Value of Money: The value of money changes with time because 
money has earning power. This is considered when comparing 
alternatives whose expenses are incurred at different rates over various 
periods of time by estimating both costs in terms of “present value” 
dollars. 

• Discount Rates: Costs that can be deferred into the future can be 
discounted because a smaller amoimt of money could be invested 
today and earn interest to make the future payment. 

• Constant versus Current Dollars: The purchasing power (“current 
value”) of the dollar varies with the general price level and inflation 
rate. Constant dollars reflect the purchasing power of the dollar in a 
selected base year. 

• Residual Value of Assets: The value, if any, left after a system has 
completed its life. 

• Indirect Benefits: Benefits that may occur beyond the intended scope 
of training. For example, the value of training to military persoimel in 
preparing them for a civilian occupation (Simpson, 1995). 

2. Common Cost Factors 


Reynolds and Iwinski (1996) state that the most common cost factors that affect 
the development of training programs, and impact on ROI include the following: 

• Instructional strategy. Technology-based strategies cost more than 
cramming students into a large room and talking at them, although 
initial up fi'ont investment costs may be easily offset by increased 
student throughput. 

• Existing course modification or new course development. (New 
courses cost more to develop, particularly in terms of instructor time.) 

• Amovmt of existing information. 

• Amoimt of individualization. 

• T 3 ^e(s) of media used (Higher technology media cost more than 
simpler technology.) 

These factors should not be evaluated individually, as they often occur in combinations, 
and thus affect the outcome of a training program accordingly. 
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The cost of training technology can be substantial, but perhaps not as severe as the 
costs associated with poor human performance within operational environments that 
involve human safety and well being. Mattoon (1998) applies a disparate set of metrics 
in his evaluation of ROI on training technologies. Several factors affecting both cost and 
those less tangible benefits of effective training include the following: 

1. Acceleration. Does the system accelerate the rate at which the learner 
acquires knowledge and skills? 

2. Automation. Does the system support the reduction of instractor workload? 

3. Availability. Does the system increase availability and accessibility of 
training for learners? 

4. Generalizability. Is the system effective for a variety of different types of 
learners? 

5. Longevity. Does the system contribute to greater retention of knowledge and 
skills? 

6. Stability. Does the system exploit current technological capabilities, emerging 
capabilities, and others that are likely to come about in the future? 

7. Strategics. Is the system applicable across multiple training environments? 

8. Transfer. Does the system increase learners’ readiness to perform in the 
operational environment? 

Mattoon’s approach to developing learner-centered design principles for 
Information Technology (IT) applications is based on a synthesis of reseaieh on learning, 
training and instruction, human factors engineering and human-computer interface 
design. His findings were used to identify what Mattoon called “General Facilitative 
Links” (GFLs). This term describes the relationships between technological capabilities 
and learning processes that can be leveraged to improve knowledge and skill 
development. The end result of such leveraging is an increase on ROI for training. The 
“General” component of a GFL principle indicates its applicability across many different 
learners and training environments. The “Facilitative” term indicates the potential 
positive impact a particular technological capability can have on knowledge and skill 
development. The “Link” component indicates the successful realization of this potential 
through effective design and implementation of Information Technologies. Each of seven 
specific GFLs are matched to the particular ROI factors discussed above. Table 3.1 
illustrates this relationship between the GFLs and the ROI cost factors. 
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GFL 


ROI Factors 

1. Instructional Simulation 

Combines instructional intervention with dynamic 
representations of conceptual material and criterion 
tasks. 

1,2,4,5,6,8 

2. Hyperlinked Content 

Links words, examples, descriptions, and other 
components of content information together into a 
structure that depicts the type of mental model 
learners are intended to develop. 

1,2, 3, 6,7 

3. Distributed Training 

Employs network technologies to increase the 
availability and accessibility of instruction, 
proficiency profiles, and other training materials 
and tools. 

1,2, 3, 6,7 

4. Verbal-Visual Pairing 

Combines abstract and concrete representations of 
concepts, factual material, task components, and 
relevant factors to facilitate information encoding 
and retention of knowledge. 

4, 5, 6, 7 

5. Multi-modal Delivery 

Presents information in at least two modes (e.g. 
audio dialogue and visual imagery) to activate 
selective attention and provide for simultaneous 
processing of information. 

1,2,4, 5, 6,7,8 

6. Proficiency Tracking 

Monitors learner interactions, captures data, 
analyzes performance, and summarizes individual 
progress and instmctional needs within Proficiency 
Profiles 

1, 2,4, 6, 7 

7. Instructional Control 

Uses Proficiency Profile information to 
automatically adapt instmction, provide advice that 
helps learners control instmction, or regulates 
shared control between learner and the IT program. 

1, 2, 3,4, 6, 7, 8 


Table 3.1 General Facilitative Links Matched to Instructional Technology Capabilities and ROI 

Factors from Mattoon (1998) 


There are many potential GFLs that can be identified to help promote the design 
of more effective training. Because GFLs represent general principles, continued 
identification of new GFLs is likely to continue and lead to one of two outcomes: (a) the 
range of applicability of GFLs will begin to overlap to the extent that individual 
distinction among them becomes academic rather than practical; or (b) the degree of 
generality or breadth of application of each GFL will become more specific. According 
to Mattoon (1998), the latter outcome is more preferable, because it leads to the 
identification and application of “Specific Facilitative Links” (SFLs). SFLs would 
provide more detailed and clearly defined boundaries for application to specific training 
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curriculum and challenges. The same logic holds for the identified ROI Factors that may 
be constrained to specific priorities, cost drivers and goals of a particular training 
operation. This degree of specificity is necessary before cost-benefit analysis can be 
effectively applied to training technologies to accurately predict ROI (Mattoon, 1998) 

F. MEASURES OF EFFECTIVENESS FOR ROI 

In any orderly process designed to provide a solution to a stated need, there 
should always be a provision for answering the question “How do I know if I have 
succeeded?” The way to do this is to establish measures against which the solution(s) 
will be tested and evaluated (Sproles, 1997). When used in conjunction with test and 
evaluation, measures are units for the quantification of qualities of the system or entity 
under review. Based on extensive research conducted by Donald Kirkpatrick (1994), the 
U.S. Navy has applied their own version of Measures of Effectiveness (MOE) to training 
technologies to better gauge their probability of success. According to a draft CNET 
Instruction on ROI for training program investments (CNET INSTRUCTION 3920. IJ, 1 
April 96), the U.S. Navy recognizes four different levels of ROI analysis that can be 
conducted when evaluating a potential program. Student and instructor reaction is the 
first level of analysis and is also the most commonly used method of evaluating training 
effectiveness; so much so, that it is considered a default MOE. Student learning 
achievement is the second level of training effectiveness evaluation. Student learning 
achievement is the most important quality indicator; however, student learning is 
influenced by a large number of external variables that are viewed as outside the trainer’s 
control (e.g., attitude, motivation, aptitude and transfer environment). Consequently, 
measures of learning are most reliable when they are taken inside the system in which the 
trainer does exercise control (e.g., in formal schools versus in the fleet). Evaluations of 
student performance in the fleet thus encompass the third level of training effectiveness 
evaluation. It should be noted that this type of performance analysis is extremely 
expensive to conduct correctly, as it attempts to control for variables that in many cases 
have nothing to do with training quality in the schoolhouse, and is therefore considered 
highly unreliable. These first three levels together make up what is termed the quality 
improvement measures of effectiveness. Specific factors that comprise this MOE include 
the following: 
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1. Quality Improvement Measures Of Effectiveness 

a. Final course test scores 

b. Follow-on course test scores 

c. Reductions in academic review boards 

d. Fleet returnee feedback (Subjective, Qualitative) 

e. Student course critiques (Subjective, Qualitative) 

The next level of analysis forms the core of the CNET Training Technology ROI 
procedures. These measures address the business results and financial impact of training 
technology investments. According to the instruction, progress against the upcoming 
Program Objective Memorandum (POM) Individual’s Account reduction goals are 
measured against timesaving measures of effectiveness. Direct cost savings are measured 
using cost savings measures of effectiveness. 

2. Time Savings Measures Of Effectiveness 

a. Reduced Individual’s Account expenditures 

(1) Implemented reductions in course length 

(2) Reductions in average on board (AOB) resulting from: 

(a) Reductions in students awaiting training 

(b) Reductions in students under instruction time 

(c) Reductions in interrupted instruction time 

(d) Reductions in students awaiting transfer time 

b. Reductions in attrition 

c. Reductions in setbacks 

d. Reduced TAD/TDY transit time 

e. Reductions in instructor personalization and course revision time 

3. Cost Savings Measures of Effectiveness 

a. Reductions in training consumables 

b. Reductions in overall TAD/TDY/PCS costs 

Based on these direct and simplified MOEs, the Navy’s intent is to provide an easily 
xmderstandable, systematic and consistent management tool whereby the Chief of Naval 
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Operations (CNO) and CNET can measure the efficiency of training technology 
investments and thus compare progress against predetermined budget reduction goals. 

G. CHAPTER SUMMARY 

Distance learning is not just about technology. It is not a singular strategy, but is 
a part of an integrated strategy for training. The learner, not the technology should be 
the focus of effort. This concept is fundamental, so when assessing returns one should 
concentrate less on the costs to the trainer and more on the benefits to the trainee. As 
previously mentioned, however, these benefits may sometimes be difficult to accurately 
measure. 

Return On Investment for a military service course is especially difficult to define 
and compute for a number of reasons. Many relevant factors are not easily quantifiable in 
monetary terms. Moreover, returns are not always clearly stated in monetary terms. The 
one benefit touted most often is that DL saves student transportation and per diem costs. 
The most interesting data found by Belcher (1999), however, was not that DL courses 
save money (although most do), but that they provide other benefits. These other benefits 
range fi-om time saved to improved training. Factors like time-to-train, flexibility of 
usage and impact on imit readiness should weigh heavily in a student-centered 
assessment. 

Drawing “before and after” conclusions about the introduction of DL into a 
curriculum can prove complicated because costs directly related to converting a course to 
a DL format are difficult to isolate. Reasons for this include the fact that many 
organizations that convert courses to DL formats often use the same opportunity to 
update or change the course’s program of instruction (POI). The resultant course may be 
completely different from the original course. If these course changes were predictable 
and consistent, then they could be incorporated into ROI estimates a priori. 
Unfortunately, they are most often not, so estimating the final ROI traceable to the 
technology insertion is difficult. Thus many course conversion costs and funding sources 
are hard to isolate. Course modification often is coincident with DL conversion, while 
some conversions are actually new courses or only partial conversions. Courses may not 
be completely converted. In some instances, schools have chosen to convert only 
portions of a POI to DL format. They believe it is necessary to maintain some of the 
classroom instruction because of the nature of the particular training (Belcher, 1999). 
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Some courses are designed from scratch as DL courses. In those instances, of course, 
there is no training “before” state with which to compare the benefits of the new course 
(at least, none that would not be realized, at least partially, by instituting a classroom- 
based course). The components of a DL conversion may be the products of multiple 
contributory resources or from different pots of money. For instance, an Army VTT 
course may be the beneficiary of proponent school courseware funding, a training support 
center paper, or CD-ROM read ahead materials. Additional competing sponsors/users 
may include the Training and Doctrine Command, the Army Distance Learning Program 
Manager, National Guard-funded VTT classrooms, or even DoD-funded satellite time 
each with their own agenda (Belcher, 1999). Finally, much data relevant to determining 
ROI are not collected. Success stories often collect data on only one aspect of training 
(e.g., reduction in student training hours) without tracking the cost of development, cost 
of fielding, cost of operations and maintenance or student test results. 

To provide the reader with a better perspective and imderstanding of the numerous 
factors pertinent to cost analysis of training and education programs, this chapter focused 
on specific issues that have been foimd to impact on ROI for DL courses. Information is 
provided briefly outlining the differences and appropriateness of CBA versus CEA in 
evaluating alternatives, followed by a discussion of suitable cost-benefit analysis 
techniques and common terminology. Although some studies conclude that there is no 
significant difference in the ultimate learning effectiveness between classroom insfruction 
and Advanced Distributed Learning-enhanced instruction (Russell, 1999), the issue is not 
a foregone conclusion. Much debate continues over this contentious point, particularly in 
regard to the MOEs that may be used in an analysis. In spite of this disagreement, 
however, there are clearly identified increases in efficiency that if applicable, should be 
included in any assessment of DL programs. Accordingly a list of MOEs commonly used 
by the U.S. Navy for training and education programs was included as well. 
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IV. PRELIMINARY OBSERVATIONS 


A. METHODOLOGY 

To gain a perspective of how DL initiatives and Interactive Multimedia 
Instruction (IMI) technologies have actually been applied to date by the USMC, I chose 
to review the curricula of selected courses currently taught at the Marine Corps 
Communications and Electronics School (MCCES), located at the Marine Corps Air 
Ground Combat Center (MCAGCC), 29 Palms, California. My purpose for choosing this 
particular location was based on the following criteria: 

(1) MCCES presently has one of the most advanced, state of the art training 
facilities in terms of implemented training technologies, as outlined in the 
USMC’s Strategic Roadmap for DL and CBT. 

(2) The variety of courses taught at MCCES are among the most lengthy and 
challenging within the USMC, ranging from several weeks to over a year in 
duration. 

(3) The quality of the instruction provides skills that are in high demand in both 
the USMC and civilian industry. Turnover in these particular technical MOSs 
is frequent, resulting in relatively high and consistent student throughput. 

(4) While functioning as an FLC, MCCES has been further designated as one of 
only two RDCs that will support all training development for the West Coast. 
Moreover, with the installation of a TEPOP server, MCAGCC serves as an 
ALC, and additionally has a fully functional VTT facility. In short, all the 
critical components of the USMC’s DL program are in place at this single 
location. 

As previously discussed, the four categories of learners found within the USMC’s 
organizational hierarchy consist of the following: 

(1) Initial skills training, to include recruit training. Marine Combat Training, and 
MOS school. 

(2) Advanced skills training. 

(3) Professional Military Education. 

(4) Marines Awaiting Training. 


For the purpose of this paper, I chose to limit my research to categories (1), (2) 
and (4). Debate continues over the applicability of DL initiatives to entry level skills 
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training. I therefore conducted my analysis of two training companies within MCCES; 
Company A, which trains technicians and maintenance personnel, and Company B, 
which trains equipment operators. Within each of the companies, I further focused my 
research on an entry-level course and an advanced skills level course, looking for specific 
problem areas. 

The courses reviewed in Company A were the Basic Electronic Repair Course, an 
introductory level course designed to provide students with elementary concepts of 
electricity, electronics, digital logic, computer operation and basic electronic construction 
techniques. This initial instruction provides the foundation for further training in the 
maintenance of telecommunications or electronics equipment and eventual qualification 
in MOSs 2800 or 5900. This course must first be successfully completed before the 
student can progress to the next phase of training. I next looked at the Technician Theory 
Course, which consists of advanced level training for senior Marines, in which students 
are introduced to sophisticated aspects of transmission theory and applications and 
provided instruction in advanced digital technology. 

Following a similar approach in Company B, I reviewed the Basic Field 
Wireman’s Course, which provides entry-level instruction in the operation of analog and 
digital wire communications equipment. I then looked at the Communications Systems 
Chief Course (CSCC) and the Operations Communications Chief Course (OCCC). The 
CSCC provides advanced level instruction in the three primary communications systems 
that the USMC currently uses; radio, wire and data networks; while the OCCC trains 
students in communications planning, including information on the organization and 
employment of USMC command and control systems, as well as routine communications 
support. 

I looked at such factors as total student throughput, failure and remediation rates, 
changes in interactive-engagement methods, student-to-instractor ratios, reductions in 
overall course hours and the potential for DL applications for each course both before and 
after the inclusion of CBT and IMI technologies. To gain a better understanding of these 
issues, I interviewed over two dozen instructors and administrative personnel, officers, 
civilians and enlisted alike, Jfrom both of the training companies, as well as support 
personnel fi'om the MCCES staff I also questioned the system administrators who 
supervised the LRC at MCAGCC. The experience levels of the interviewees varied, with 
some individuals having been present at the schoolhouse for a couple of years, both 
before and after the implementation of CBT in the classroom, while others had only just 
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recently arrived. The variance in experience levels, combined with the degree of 
seniority and knowledge of the personnel involved, provided an accurate representation 
of the differing opinions regarding the inclusion of CBT in the classroom, allowing me to 
accurately capture many of the concerns surrounding training technology issues. I further 
reviewed the problem of Marines Awaiting Training, focusing on how information 
technologies, to include DL, have addressed this issue. Finally, I examined the use of the 
LRCs, both at the schoolhouse and the base, to identify any trends in the levels of use 
since they were opened. 

B. OVERVIEW OF MCCES 

The training constraints currently faced by MCCES originate from the guidance 
provided by former Commandant, General Charles Krulak (Krulak, 1995), for all formal 
schools to reduce the time Marines spend in entry level training, in order to get them to 
operational units faster, thus reducing the overall number of Marines in the T2P2 account. 
The more time junior Marines languish in a training status, the less return the Marine 
Corps can expect from its recruiting investment. 

Based on the personal anecdotes of instructors and administrators, the plaimed 
reductions in individual course length, for example, were not carefully considered, in 
many cases resulting in arbitrary reductions in various courses. Procedures for formal 
course revisions, as outlined in the Systems Approach to Training (SAT) manual, were 
ignored. Input from Subject Matter Experts (SMEs) regarding the retention or 
elimination of Individual Training Standards (ITSs) from a particular course of 
instruction, and normally reviewed during a Course Curriculum Review Board (CCRB), 
was not requested. In some cases, the training companies were directed to reduce overall 
course lengths by an across-the-board percentage without regard for what would be 
eliminated. It was felt that new CBT and DL technologies would increase the overall 
efficiency of the learning process, as conventional classroom time was reduced. As a 
consequence, however, some courses were cut too deeply with reductions as great as 50 
percent of critical ITSs completely removed. When asked for their opinion on whether 
CBT has made up for the reductions in conventional classroom time, responses were 
mixed. Senior instructors felt that CBT, although useful, was not a substitute for the 
mandated reductions in conventional classroom time. Junior instructors favored CBT and 
felt that it improved overall instruction in terms of learning transfer and retention. These 
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junior instructors had been teaching prior to the activation of the Automated Electronic 
Classroom (AEC), and had the benefit of conventional classroom experience. 

Instructors reported that frequent complaints were received by MCCES from 
operational units regarding the training of junior Marines arriving from the schoolhouse 
since the activation of the AECs. Taking into the accoimt the experience level of senior 
non-commissioned officers in the operational units, senior instructors and administrators 
from the schoolhouse determined that most of the complaints were unfounded and based 
mainly on conjecture, as the experience levels of junior Marines arriving prior to the use 
of CBT were never evaluated. No baseline therefore existed against which to make such 
claims. 

The current period under observation (Winter/Spring) was considered the busiest 
time of the year in terms of student throughput for the schoolhouse. Larger numbers of 
Marines recruited during the summer months (after graduation from high school) having 
completed recruit training and Marine Combat Training (MCT), were now receiving 
entry level MOS training. In accordance with the TEMI, the fielding plan for the 
acquisition, installation and operation of an integrated telecommunications system for 
distribution of IMI and training software was on schedule. The hardware and software 
components of MCCES and MCAGCC are currently in place and fully operational. 
Accompanying changes in the traditional methods of instruction to include course length, 
curriculum content and delivery mechanisms had only recently been completed and 
implemented the preceding summer. Little had been done prior to my visit to implement 
formal DL solutions to improve instructional effectiveness. There was unanimous 
agreement that DL technologies should not be applied to entry level skills training. They 
should be used for advanced skills (core plus) level courses. 

C. COMPANY A 

Company A provides training for technicians and maintenance personnel. The 
entry-level course reviewed was the Basic Electronic Repair Course (Course ID# 
M092721). Course length is approximately 40 training days. Class capacity ranges from 
a minimum of 15 students to a maximum of 30 students. There are 50 classes held per 
year with an average annual throughput of 1300 Marines. This figure is in accordance 
with the Training Input Plan (TIP), which estimates an annual throughput of 
approximately 1325 Marines for FYOl. It should be remembered that this is a feeder 


40 



course for further entry level training, so depending upon the individual Marine’s 
particular MOS (either 2800 or 5900 field), additional training time may last over a year 
(Herring, 2000). 

1. LABVOLT Experiment 

This course was of particular interest to my study due to the use of some 
innovative automated electronic training technologies that had been recently instituted. 
Known as LABVOLT, this technical training system features educationally substantive, 
competency-based curriculum providing hands-on activities for learning, testing, 
troubleshooting, applying and designing basic-to-advanced electronic circuits. The 
system consists of a breadboard of basic electronic circuits connected to an interactive 
computer program designed to walk the student through each lesson. The system offers 
stand-alone capability in which the student can progress at their own pace, as well as a 
test and remediation package that guides the student through each progressive skill level. 
Based on discussions with course instructors, this particular system was designed for 
civilian schools and has been used with success by the ITT Technical Institute and DeVry 
Institute of Technology. It was pointed out that many of these institutions furnished only 
one computer for two to three students. This was felt to be inadequate for this type of 
instmction. Accordingly, the number of students per class for the Basic Electronic Repair 
Course was capped at 27 to ensure a terminal-to-student ratio of 1:1 (Weber, 2000). 

A study at the Marine Corps Training and Evaluation Branch revealed that since 
the application of this technology, the overall coiuse length was reduced from 55 to 40 
days, but most interestingly, the overall recycle/failure rate was reduced from 
approximately 46 percent to 6 percent. Upon closer inquiry with instructors and staff, it 
was revealed that the results were even more impressive. Under the traditional lecture 
method, approximately 80 percent of the students were set back at least once during the 
11-week course. After the full implementation of the LABVOLT curriculum, setbacks 
were reduced to 15 percent. The attrition rate remained about the same. As impressive as 
these results were, however, it was soon discovered that although the system was 
designed to stand alone, after-action comments fi'om students indicated that difficulties 
were still being encountered in terms of learning transfer and retention. As indicated in 
the Program of Instruction (POI) for this course, the number of instractors per class was 
two. Apparently, while the instmctors were busy answering questions and helping the 
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more vocal students experiencing difficulty, others would grow impatient and simply 
press on through the program without waiting to have their questions answered. This 
problem came to light when instructors later went back and checked the progress of the 
students. The program was designed in such a way so as to see exactly where each 
individual student experienced difficulty, if any, by the way in which the questions were 
answered (DeLeon, 2000). 

As a result of these initial experiences with the LAB VOLT system, it was decided 
to reintroduce lectures back into the POI to aid in retention of basic information. This 
action not only increased retention, but also actually allowed the students to progress 
through the lessons at a faster rate, so no time was lost in the overall schedule. The 
instructors felt that the use of such lectures prior to the start of each period of instruction 
oriented the students and enabled them to better understand and comprehend key 
concepts and ideas as they continued independently. It was also found that the overall 
number of questions significantly decreased when the lectures were reintroduced (Weber, 
2000 ). 


2. Marines Awaiting Training 

Another area of interest concerned how Company A had addressed the issue of 
Marines Awaiting Training (MAT). Through the use of such technology as outlined 
above, the Company was able to reduce its MAT status from an average of four months 
to an average of 30 days (An overall reduction of 75 percent). Much of this reduction 
was attributed to the faster progression rates the students experienced as a result of the 
LAB VOLT system. Upon closer examination, however, it was formd that much of this 
reduction in MAT was achieved by conducting the course 24 homs/day, in shifts, with 
half the instractors providing instruction dxxring the day, and the other half teaching at 
night. Although successful in significantly reducing the numbers of Marines in MAT 
status, the problems encoimtered included such issues as equipment breakdown from 
overuse and lack of scheduled maintenance; instructor burnout and exhaustion; and most 
importantly, rather than being eliminated, the MAT issue was only shifted to a follow-on 
phase of the Marine’s training, as the other courses could not handle such large 
throughput at the same rate. As a result, the number of courses per year was reduced to 
current levels (DeLeon, 2000). 
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An alternative approach to this problem used by Company A was to provide the 
Marines in MAT with follow-on training that had previously taken place within the first 
six months that the Marines arrived at their operational units. Training included events 
such as uniform refitting, gear accountability and inspections, physical fitness, drill, core 
value instruction, sexual harassment training, etc. Not only does such activity keep the 
Marines suitably occupied, but also allows the Marine to be put directly to work upon 
reaching his permanent duty station. This approach has received favorable endorsement 
and feedback from operational units. Unfortunately, this technique was not in practice 
throughout MCCES, as each training company exercises its own method of handling the 
MAT issue. Current figures for the Basic Electronic Repair Course at the time of this 
research consisted of 434 Marines currently in formal training, 152 in MAT status 
(informal training), and 10 personnel-awaiting disposition (i.e., possible 
failures/recycles). Obviously, the issue of MAT was not a problem for the advanced 
skills level courses due to smaller student throughput and scheduling practices. 

3. Technician Theory Course 

The Technician Theory Course (TTC) (Course ID # M09TA31) is approximately 
80 training days in length. As mentioned, it is an advanced skill level training course 
designed to provide instruction to Marines in the grade of Corporal through Gunnery 
Sergeant on a second or subsequent enlistment. According to the Course Descriptive 
Data (CDD) sheet, student capacity ranges from a minimum of 12 students to a maximum 
of 24 students. There are 10 classes held per year with a total student throughput of 188 
Marines per year, which is in accordance with the TEP. Each individual class averages 24 
Marines. This course has also been fully converted for use in an Automated Electronic 
Classroom (AEC) environment, although the specific lessons are not fully interactive as 
in the LABVOLT example. As an advanced skill level course, providing core-plus 
training, DL applications were viewed favorably by instructors and administrators alike. 
Based on interviews with schoolhouse personnel, it was determined that as much as 50 
percent of the current curriculum could be disseminated via DL technologies, thus 
reducing overall course length significantly (Herring, 2000). Not only would prospective 
students be able to review and validate much of the introductory information, they would 
begin the course on a relatively level playing field in terms of knowledge and thus be able 
to absorb follow-on material much more quickly. 
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A word of caution is in order here, however, regarding course length reductions 
based on predicted savings from DL applications and other automated training 
technologies. One of the greatest frustrations expressed by instructors and administrators 
alike was the inclination of HQMC to mandate reductions in overall course lengths based 
on predicted savings from advanced training technologies. Particularly in regard to the 
entry-level courses, it was felt that time saved by the introduction of automation into the 
classroom should be used to provide additional training rather than eliminated from the 
curriculum. The reasoning was that junior Marines require as much exposiue as possible, 
to the numerous systems currently entering the USMC’s inventory that they will be 
required to operate and maintain. Under the current guidance to reduce training to the 
absolute minimum and get the junior Marine to the fleet as soon as possible, this is not 
taking place. Hence, the complaints from operational units that the overall quality of new 
Marines arriving from MCCES has dropped. This argument was not found to carry equal 
weight in regard to advanced skills training, however, as the key with more senior, 
experienced Marines, is to keep them in the operational imits as much as possible where 
their expertise is needed. Advanced skill level instruction is a training niche for which 
DL seems exceptionally well suited. 

Accordingly, measures are currently being developed to adopt this course to a DL 
format, although no specific date was set for this to occur. The predicted savings from 
such action were expected to be significant, based on the mythical training day cost of 
$271 per Marine. I enquired into the source of this $271 figure, seeking to determine 
where it came from, and upon what assumptions it was based. No one within Company 
A or MCCES could explain where it originated, or upon what assumptions it is based. 
Nonetheless, it is used frequently for budgeting purposes. It would therefore appear that 
although the estimated savings from the introduction of DL technology into this 
particular course may be substantial, the actual value will depend upon a determination of 
what it actually costs to train a Marine per day for this particular course. In all fairness to 
MCCES, however, it should be noted that using antiquated or inaccurate costing data is a 
convenient practice that occurs throughout military and civilian budgeting (Gramlich, 
1998). 
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D. COMPANY B 


Company B provides both entry-level and advanced skill training for 
communications systems operators. The Basic Field Wireman Course (Course ID# 
M092471), which was examined as an entry-level program, specifically furnishes 
instruction in the operation of Fleet Marine Force analog and digital wire 
communications equipment. It encompasses the application, installation, adjustment and 
familiarity with the operational characteristics and employment of wire/telephone 
equipment. Additional requirements of this course include demonstrated proficiency in 
communication procedures, and basic knowledge of relevant publications, directives and 
security regulations. Course length is approximately 17 training days. Class capacity 
ranges from a minimum of 20 students to a maximum of 40. There are 16 classes 
convened per year with an average annual throughput of 640 Marines, which in this 
instance was in excess of the TIP requirement of 596. The discrepancy can be attributed 
to the fact that the TIP has not yet been updated for this particular course, which was 
reduced in length from 40 to 17 training days. 

1. Basic Field Wireman’s Course 

This reduction in overall course length represented perhaps the most extreme 
example that was found regarding the application of CBT in the traditional classroom 
setting. As such, there was also the most diverse range of opinions regarding the success 
of this action. There was general agreement among all the instructors, both young and 
old, that the course had indeed been cut too severely, leaving no time to teach many of 
the critical tasks and skills needed by the Marines to successfully function in their 
operational billets. For example, one of the key tasks omitted imder the new 
reorganization was that of commimications procedures. According to the instructor staff, 
when directed to implement overall course reductions in order to speed up the training 
process, the chain of command arbitrarily decided that these skills would be eliminated 
without regard for the advice of SMEs. It was determined that these skills could be 
acquired on the job, once the Marines reached their operational units. This was clearly 
another example of where the SAT process was completely ignored and key skills were 
eliminated in the interest of saving time. Although the intent may have been sound, this 
action was flawed because the advice and experience of the SMEs was ignored, thus 
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resulting in inadequately trained Marines sent to FMF units. The problem was confirmed 
by feedback from operational units, which stated that many of the field wiremen, upon 
reaching the fleet, were found lacking in basic skills normally expected of their billet and 
rank (Lewis, 2000). As such, time had to be taken away from other events by the 
operational units to train these Marines up to the expected level of proficiency. 

2. Increased Training Requirements 

Another area of concern within this course dealt with the introduction of new 
communications equipment within the Marine Corps inventory. Many of the new 
systems being fielded will function as key components of vast communications networks, 
and will be the responsibility of field wiremen to install, maintain and operate. 
According to one instructor, “the days of simply running wire and climbing telephone 
poles are over!” The result of this vast technology explosion is that new equipment is 
being introduced as complements rather than replacements for existing equipment. A 
review of the Course Descriptive Data fmmd that the students were currently responsible 
for gaining proficiency with not less than ten major sets of commimications devices, to 
include accompanying cables, power units and assorted support equipment. The munber 
of systems for which field wiremen are now responsible is therefore increasing rather 
than decreasing, yet the response of HQMC to this phenomenon is to cut formal training 
rather than increase it. Even with the introduction of CBT into the classroom, the 
instructor staff was in consensus that there is still insufficient time currently available to 
teach the Marines everything they should know to function proficiently in their basic 
MOS. Additionally, it was explained that due to the reorganization and consolidation of 
certain MOSs within the communications field (2514 with 2512, for example), the 
smdents were now required to learn additional skills that they had previously not been 
held responsible for at that point in their careers (Rice, 2000). Such requirements have 
only compounded the problem of insufficient time to teach a myriad of skills and tasks. 

3. Automated Electronic Classroom 

There were distinctly mixed opinions regarding the application of CBT in the 
classroom. To support the training effort, the facilities available included an 80-man and 
50-man capacity AEC, each providing individual workstations with self-paced. 
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interactive media work packages. The older, more senior instructors felt the CBT 
technology had some merit, but that too much emphasis was being placed on this 
equipment to take the place of traditional student/instructor interaction. In the opinion of 
these instructors, overall lecture and demonstration time had been significantly reduced, 
impacting on student comprehension. The yoimger instructors, however, found that 
under the present time constraints, the introduction of CBT into the classroom was the 
only feasible solution to successfully accomplish their jobs. A review of student 
performance, in terms of test scores and completion rates, provided some measure of how 
the students were reacting to the new technology. Test averages from three random 
classes that graduated prior to the activation of the AEC (which occurred in early 1999) 
totaled approximately 89.39 percent, while those of three random classes that followed 
totaled 95.14 percent. Granted that this AEC application is still in its infancy and 
insufficient data currently exist for a more thorough and detailed study, these early 
indications support claims made by various studies that the use of CBT results in no 
significant change in student comprehension and learning (Russell, 1999 and Mattoon, 
1997). It was recognized, however, that even the junior instructors preferred more lecture 
and practical application time when possible, finding that similar to the LAB VOLT 
example in Company A, overall remediation rates decreased, while the rate of student 
comprehension and progression increased proportionately. It seems the issue of 
timesaving though the use of CBT and other automated training technologies was being 
addressed as previously discussed. Rather than providing more time to devote to other 
requirements, the saved time was simply being eliminated from the POI, requiring the 
instructors to do more with less. 

4. Advanced Level Training 

Advanced skill training was provided in Company B in the Communications 
Systems Chief Course (CSCC), and the Operations Communications Chief Course 
(OCCC). The CSCC was conducted three times per year with an average throughput of 
120 Marines per year. The OCCC held eight classes per year, consisting of four wire and 
four radio sub-courses for an average annual throughput of 216 Marines. As part of 
overall course reductions, the CSCC was shortened from 12 weeks to nine weeks, while 
the OCCC was cut from 16 weeks to 12 weeks. The CSCC was significantly reorganized 
after the introduction of the AEC, as part of the curriculum package. Each course 
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consisted of two subclasses focusing on radio and wire networks respectively. Previously 
the POI had consisted of only a single class, divided into thirds with an equal amount of 
time spent on radio, wire and data information systems. The instructors who taught these 
courses were the most senior instructors within MCCES, having a wealth of both 
operational and training experience. Their perceptions of the use of CBT within the 
classroom were generally positive, although they too were in agreement with many of the 
other school personnel that the time savings realized from the use of automated training 
technologies were simply being eliminated, rather than used for additional skill 
development. Their observations were that while the overall quality of the students was 
excellent, general experience levels were significantly lower due to more rapid promotion 
practices. This situation thus required more time to instill knowledge that in the past the 
average student, at that stage of their career, would have taken for granted. General 
computer skills were also found to be lacking among students in both the CSCC and the 
OCCC, requiring additional instructor time to familiarize students with the basics of word 
processing and graphics programs. The particular period of instruction that I observed 
received basic instruction in the use of Microsoft PowerPoint presentations. This skill 
was a necessary part of the curriculum requirement, used to illustrate various diagrams of 
communications networks and plans. Based on discussions with instructor personnel, the 
general student experience level with such computer programs prior to attending school 
was moderate, but getting better. 

Similar reviews of test scores and overall class averages both before and after the 
introduction of the AEC, were conducted to gain a better perspective of student 
performance. A random sampling of three classes prior to the use of the AEC resulted in 
an overall student average of 92.73 percent, while the cumulative average of three 
random classes after the use of the AEC was found to be 91.10 percent. The overall 
failure rate of Marines for these particular courses was so insignificant, both before and 
after the use of the AEC, that it was not even considered. Once again, it is acknowledged 
that the while the use of the new training technology is so recent that sufficient data has 
not yet been compiled for a more detailed analysis, the overall differences appear to be 
minimal at best. Numerous other studies in the field of civilian industry and academia 
bear out this “no significant difference” phenomenon (Russell, 1999). 
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E. THE DL TRAINING NICHE 

Based on the increased demands of the advanced level courses, combined with 
overall reductions in training time, instructors felt that future students required 
demonstrated proficiency in certain basic skills, prior to arrival at the schoolhouse in 
order to be successful. Distance Learning technologies were seen as being most useful. 
By reviewing and refreshing certain core concepts and familiarizing themselves with 
basic computer processing skills, it was felt that precious time spent on these events could 
be eliminated. This would allow for more time devoted to advanced learning. As part of 
this initiative, the staff was currently working on a number of initiatives, to include the 
use of MCI courses, web-based training and other basic skill preassessment techniques 
that would allow students to validate as much of the introductory information as possible, 
while still at their parent units. Thus upon their arrival at the schoolhouse, the experience 
level of the students would be uniform, allowing more time to be devoted to advanced 
concepts and techniques. The instructors were quick to point out that in order to be 
successful, however, certain criteria had to be observed. First, the additional timesavings 
provided by the use of DL must not be eliminated from the curriculum. Rather, they 
should be used to provide the additional skills required for the student’s rank and 
experience level. Second, prior to attending the reduced formal portion of the course, the 
prospective students must be given adequate time, while still at their units, to complete 
the DL portion of the instruction free from the daily distractions of their regular duties. 
Failure to observe either of these prescripts would not realize any appreciable savings 
from the use of DL technologies. 

F. SUBJECT MATTER EXPERTS 

Another problem area dealt with changes in course curricula based on the infusion 
of the new technology, both into the classroom and into operational imits. Similar to 
students in entry-level courses, senior Marines are expected to be proficient with the new 
communications systems continuously being fielded to operational units. The principal 
concern is from where the training on these new systems for senior Marines is to come. 
Should they be trained at their imits or at the schoolhouse? According to the SAT 
process and the more recent TEMI, changes in formal school course curricula must be 
submitted, discussed and approved at a formal annual CCRB. Subject Matter Experts 
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from the pertinent occupational fields meet and decide which ITSs will be taught at the 
schoolhouse and which will be disseminated at the local unit level. The problem is that 
many of the designated SMEs are not SMEs at all. Changes in technology and new 
equipment occur so rapidly that many of the senior Marines chosen as SMEs are 
unfamiliar with new technologies and the requirements for proficiency with them. As 
such, they aggressively push to conduct the majority of the training with this new 
equipment at the schoolhouse. These same individuals are found to be the ones who then 
complain when new Marines arrive at their units inadequately trained to operate the 
equipment. Schoolhouse personnel felt that to properly deal with this problem, SMEs 
should not be selected solely on the basis of seniority. An individual’s work history 
should be considered as well (Foy, 2000). More importantly, the occurrence of such 
situations requires a fundamental change in the way the Marine Corps views training. The 
SAT and the TEMI are good systems, provided the USMC embraces them and doesn’t 
pay them mere lipservice. 

G. LEARNING RESOUCE CENTERS 

Both MCCES and MCAGCC operate and maintain LRCs for use by students and 
permanent persormel alike. The ABCs located at the schoolhouse were dual hatted as 
LRCs when not being used for formal classes, while the base maintained a functional 
LRC comprised of 20 workstations with Internet and Intranet capability. I was 
specifically interested in looking at usage rates since the introduction of these facilities in 
mid-1999. Reviewing the logbook for both LRCs, I made the following observations. 

The principal categories of usage for the LRCs consisted of education-related topics such 
as web-based MCI courses, personal e-mail and general Internet applications. Table 4.1 
illustrates that from the period of Jime 29, 1999 when the facilities first opened to 
December 15, 1999, the average daily use rates in terms of people per day for the 
MCCES AEC/LRC were as follows: 


CATEGORY 

Usage Rate (People per Day) 

1) Education 

00.45 

2) Personal e-mail 

10.27 

3) General Internet 

3.00 


Table 4.1 MCCES AEC/LRC Average Daily Use Rates from 29 Jun 99 -15 Dec 99 



Inspection of the logbook for the base LRC revealed similar values. This period covers a 
time span of approximately 62 days, not to include weekends and holidays. After 
December 15, 1999 personnel were no longer granted personal e-mail access due to 
increased security measures and bandwidth redesignation. For the period from December 
16, 1999 to January 19, 2000, usage rates plummeted to less than one individual per day 
for both LRCs. The principal use of these facilities made by personnel had apparently 
been for personal e-mail access. This second finding came as no real surprise for the 
MCAGCC LRC, based on initial research at the schoolhouse AEC. An equally 
contributing factor was a series of directives issued by the base Commanding General, 
who as part of his quality of life initiative, had previously ordered that the LRC be made 
available to all dependents to contact their spouses while on deployment. 

Staff personnel at the schoolhouse were not of the opinion that the AEC would 
receive much use by students in their off duty hours for DL applications. It was felt that 
the last thing students would be most likely to do after a full day of formal instruction 
would be to pursue further learning on their own. An additional consideration was that 
with reduced classroom time, more homework was being assigned than in the older, 
longer courses. Most entry-level students are so overwhelmed at this point in their 
service; they simply have no desire for career-enhancing training unrelated to their MOS. 
It was recognized that if DL applications had any relevance to entry-level training, it 
would be for Marines awaiting training. Distance Learning applications could be used as 
part of the entry-level training requirements separate fi-om the Marine’s primary MOS. 
Competing distractions would be at a minimum. They would be free to concentrate on 
DL courses. As promising as this initiative sounds, no actions have yet been taken to 
implement it. MCI currently offers only three electronic courses of instraction, which are 
as follows (Taylor, 2000): 

1) Land Navigation 

2) Fundamentals of Diesel Engines 

3) Personal Finance 

Based on discussion with the LRC facilitator, the electronic tests for these courses do not 
match the course numbers, leading to confusion for the student and the test administrator. 
Until MCI solves these problems and converts more of its courses to an electronic format, 
which it is currently doing, usage rates will remain low. 
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Perhaps the most disturbing situation I observed during my visit was that T&E 
Branch is already seeking feedback on the LRC, although insufficient time has elapsed 
for the system to prove itself Fully operational for close to nine months, the system has 
been underutilized for lack of instructional materials. Once this situation is corrected, the 
potential use of this DL application should increase for the appropriate category of users. 


H. CHAPTER SUMMARY 

The information gathered at MCCES and MCAGCC thus revealed some 
interesting inconsistencies between the plan outlined in the USMC’s strategic vision for 
the application of advanced instructional technologies, and what is actually occurring at 
the schoolhouse as well as in the operational units. Improper use of the SAT and TEMI, 
combined with flawed expectations of CBT/DL technologies, have resulted in drastic 
reductions in course lengths, often to the detriment of the quality of instraction. 
Increased training requirements caused by continuous fielding of new equipment have 
further complicated already packed training schedules, which as mentioned above, are in 
the process of being shortened. Problems with the experience level of SMEs, chosen to 
decide which tasks to teach at the schoolhouse and which to leave with operational units, 
have brought into question the overall ability and quality of the individual Marine. 
Finally, a lack of adequate instmctional materials has resulted in a gross underuse of the 
LRCs, by both students and permanent personnel. As serious as these problems are, 
solutions do exist as evidenced by the recommendations of schoolhouse personnel. The 
unanswered question is how long will these problems be allowed to manifest themselves 
before action is taken? 
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V. 


ROI METHODOLOGY 


A. COST-BENEFIT ANALYSIS PROCESS 

CBA encompasses a range of procedures and is not a single technique. Sassone 
and Schaffer (1978) contend that although CBA incorporates certain general principles, it 
is difficult to design an all-purpose CBA procedure because of differences in 
public/govemment projects. They provide a basic framework for conducting a CBA 
consisting of initial planning stages followed by data collection, separate cost and benefit 
analyses, and presentation of results. McMichael (1985) provides a similar framework. 
While there are some differences in their formulations, the authors would probably agree 
with Swope (1976) that a CBA process should include the following steps: 

• Formulate Assumptions 

• Determine Alternatives 

• Determine Costs and Benefits 

• Compare and Select Alternatives 

• Conduct Sensitivity Analysis 

Assumptions are usually made regarding what variables will affect the process 
and the range of values those variables will present. The alternatives will include the new 
system and one or more other possibilities. Frequently, one of these is an existing 
system. After the costs and benefits of alternatives have been determined, they are 
compared and a selection is made. In CBA, the best alternative is the one yielding the 
greatest net benefit (i.e., the alternative whose benefit value, expressed in monetary terms, 
less its cost is the greatest) (Simpson, 1995). Orlansky (1989) provides the following 
concrete example: 

[In] cost-benefit analysis...both the input and output values can be 
measured in monetary terms. This requires an open market to assess the 
value...of the output that results from a particular use of resources (i.e., 
the costs). One example might be a cost-benefit analysis of a particular 
form of advertising. The costs are those needed to develop and conduct a 
particular advertising program; the benefits are the profits that may be 
attributed to the advertising program (p. ix). 
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Assumptions are required in planning a CBA and these can lead to uncertainty in 
the outcomes of analyses. If the CBA is locked into a single set of assumptions with the 
intent of obtaining a definitive result, its outcome may be too firagile to be trustworthy. It 
is more sensible to vary the assumptions systematically and to provide the results of 
analyses under different assumptions. This procedure is referred to as sensitivity analysis. 

According to Simpson (1995), a CBA does not sufficiently evaluate military 
projects, however, for there is no market available to establish the monetary value of the 
output in terms of performance, although inputs can be expressed in monetary terms. He 
further states that a cost-effectiveness analysis (CEA) is the preferred method used in the 
DoD to make decisions regarding alternative courses of action where the outcomes affect 
military performance. Examples may include choosing among a set of the following: 

a. Weapons Systems 

b. Weapon System Upgrade Programs 

c. Training Methods 

A definition of CEA analogous to that given earlier for CBA might be an estimation and 
evaluation of the military value associated with alternatives for achieving defined military 
goals (Simpson, 1995). CEA is used to help meet military goals (rather than CBA’s 
public/govemment goals). CEA, like CBA, compares alternatives using a formal process. 
Criteria decide the outcome for both CEA and CBA, but the criteria differ (i.e. military 
value for CEA and public benefits for CBA) (Simpson, 1995). Economic Analysis, an 
additional term used in a number of DoD publications, has a meaning synon 3 mous with 
CEA (Rankin & Swope, 1991). 

B. COSTS AND MILITARY VALUE 

Simpson (1995) maintains that the costs of alternatives in a CEA are estimated in 
a manner similar to that of a CBA by using cost models that take into accoimt all of the 
associated costs of the alternatives through a projected life cycle. He suggests, however, 
that estimating military value for a CEA is different from estimating public benefits in a 
CBA. An important difference between CEA and CBA is that the outcome (military 
value) is not defined in the same terms as cost (Orlansky, 1989; Rankin & Swope, 1991). 
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The military value of ineffective training methods, for example, is hard to monetize. An 
instructional program that sends a poorly trained Marine to an operational unit represents 
an opportunity cost to that unit in terms of lost output. A senior, more experienced 
Marine must now be detailed from his regular duties to work with the junior Marine to 
bring the individual up to the desired level of knowledge and skill. The time and effort 
spent working with the junior Marine is a cost to the imit in terms of the senior Marine’s 
normal output as well as the junior Marine’s inefficient output. The benefit of what 
should be a high-quality training program has now become a cost. Yet how exactly does 
one associate a value with this cost (i.e., the poor-training program)? According to 
Boardman, Greenburg, Vining, and Weimer (1996), the intermediate good method can 
be used to estimate the benefit of a project (the training program) based on its value 
added to the downstream activity (i.e., a qualified Marine’s output in the unit). 
Investment in the skills and abilities of human beings improves the stock of human 
capital, and thus make individuals more productive and more valuable to the organization 
(Boardman, et al., 1996). In terms of a cost analysis, the military value of the 
intermediate good (the poor training program) has become an opportunity cost in regard 
to both the junior Marine’s lost output, and the senior Marine’s normal lost output, which 
are relevant factors that are difficult to measure. The authors argue that when dealing 
with such intermediate goods whose linkage to preferences is not clear, a CEA is a 
suitable alternative to a CBA. Orlansky (1989) said: 

[The cost-benefit] procedure cannot be followed when examining 
the products of a military weapon or training program (emphasis added). 

There is no open market that can establish the monetary value of increased 
readiness, better-trained personnel, or better weapons (p. ix). 

Simpson (1995) concludes that, ultimately military value is reflected in the degree 
of combat success. According to his analogy; weapon system A has greater military 
value than weapon system B if A is more likely to prevail in battle than B. Or, if two 
training alternatives are being compared, treatment A has greater military value than 
treatment B if A better equips students to prevail in battle than B. Military value can be 
assessed empirically only in combat and it is impractical to wait for a war to make an 
assessment. An alternative to combat is to create a combat-like environment (e.g., to use 
an instrumented live exercise). In performing CEA, measures of effectiveness (MOE) are 
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used that ostensibly predict combat success (Simpson, 1995). In the experimental 
paradigm, Simpson (1995) further defines MOEs as equivalent to dependent variables; 
the same variables used to assess the impact of an experimental treatment condition. 

Like CBA, CEA encompasses a wide range of procedures and is not a single 
technique. Because of conceptual similarities between CBA and CEA, it is reasonable to 
extend Sassone and Schaffer’s (1978) contention regarding the difficulty of designing an 
all-purpose CBA procedure to the realm of CEA. Likewise, the basic framework for 
conducting a CEA parallels that of a CBA, described by Swope (1976), but with a slight 
change to the third step (“Benefits” becomes “Military Value”): 

• Formulate Alternatives 

• Determine Alternatives 

• Determine Costs and Military Value 

• Compare and Select Alternatives 

• Conduct Sensitivity Analysis 

Since cost and military value use different units, selection of alternatives cannot be done 
on a cost basis alone as with CBA (Simpson, 1995). Orlansky (1989) has described the 
decision-making logic as illustrated below: 


COST 

EFFECTIVENESS 


LESS 

SAME 

MORE 

LESS 

UNCERTAIN 

ADOPT 

ADOPT 

SAME 

REJECT 

UNCERTAIN 

ADOPT 

MORE 

REJECT 

REJECT 

UNCERTAIN 


Table 5.1 Orlansky’s Decision Logic Diagram for Evaluating the Relative Effectiveness and 


Cost of Two Training Methods During CEA (1989) 

Orlansky (1989) commented as follows on the interpretation of the diagram. 

a. If one alternative is as effective or more effective than another and it 
costs less, adopt it; it is also the preferred choice if it is more effective 
and costs the same. 

b. If an alternative is less effective and costs the same or more than 
another to which it has been compared, reject it; this is also the case if 
it is equally effective but costs more. 

c. If any of the following combinations of the cost and effectiveness of an 
alternative is foimd, no rational preference can be made; 
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(1) less effective and less cost 

(2) equal effectiveness and equal cost 

(3) more effective and more cost 


Based on the nature of the data gathered from my observations at MCCES, the 
characteristics of CEA are particularly relevant here, so a combination of both CBA and 
CEA will be used to conduct my analysis. 

C. BENEFITS AND VALUE ADDED 

Measuring the direct benefits or results of instruction is a more complex task than 
cost identification since some benefits may be difficult to identify. The objective is to 
attach a quantitative value to each identified benefit, which must be derived from the 
instructional alternative being examined. There may be other factors that contribute to a 
given benefit. The analyst must isolate and measure only the effects of training. The 
American Society for Training and Development’s 2000 ASTD State of the Industry 
report, which outlines trends in over 60 major industries, lists the following as typical 
benefits to consider when conducting an analysis of DL technology: 

• Reduced training time. Reductions may be realized in facility, 
personnel and travel costs, as well as less nonproductive/lost time 
while the student is in training. 

• Reduced material costs. Potential cost savings for revision, 
distribution and maintenance of instructional materials/courseware in 
electronic format versus printed or “hard copy” materials. 

• Improved on-the-job safety and proHciency: Translates into fewer 
accidents and increased productivity. 

• Improved job performance. Less time to perform/accomplish tasks, 
increased productivity/efficiency/quality of work, or improved 
management/command/supervision/decision-making ability, amount 
of positive versus negative feedback from “clients”, changes in 
employee morale and motivation. 

• Reduced manpower requirements. Potential manpower savings as a 
result of increased efficiency or productivity, less supervision required, 
etc. 
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• Reduced equipment requirement. Simulation training may reduce 
the need, impact, or expense of allocating actual equipment to train 
hands-on skills. 

• Higher equipment operational availability. Fewer 

operator/maintainer errors, safety incidents/accidents, etc., may result 
in lower maintenance requirements and equipment/system downtime. 

Reduced to its most elemental form, a CBA evaluates costs versus benefits for a 
particular project. Boardman, et al., (1996), state that the basic decision rule for a single 
alternative is simple: Add up the present value of benefits (B), add up the present value 
of costs (C), and see which is larger. If benefits exceed costs, then proceed with the 
project. If not, stay with the status quo. In short, the analyst should recommend 
proceeding with the project based on the following formula: 

NPV = B-C>0 

When there is only one potential project, proceed with that project if the net present value 
of benefits (NPV) is positive. When there is more than one alternative to the status quo, 
the rule is slightly more complicated: Select the project with the highest NPV (Boardman, 
et al., 1996). 

As straightforward as this process may seem, however, problems arise in 
determining precisely what the benefits will be, and more particularly in attaching a 
monetary figure to those benefits that are identified. Thompson and Strickland (1995), 
address this issue by incorporating the concept of a value chain into the process of cost 
analysis. According to them, the primary analytical tool of strategic cost analysis is a 
value chain identifying the separate activities, fimctions and business processes 
performed in designing, producing, marketing, delivering and supporting a product or 
service. The chain starts with raw materials supply and continues on through parts and 
components production, manufacturing and assembly, wholesale distribution and 
retailing to the ultimate end user of the product or service (Thompson, et al., 1995). In 
our case, the raw material is the entry-level Marine attending basic skills training and the 
senior Marine attending advanced-level training. The ultimate end user is the Marine 
Corps, or more specifically, the operational units. The key here is to incorporate DL 
technologies in such a way as to increase the value of the Marine by enhancing his 
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learning ability and retention skills, while simultaneously reducing the amount of time 
the Marine is away from the operating forces. 

D. ANALYSIS STRATEGY 

Continuing on with Simpson’s (1995) steps of conducting a CEA, I will now 
apply them to my observations from MCCES. 

1. Formulate Assumptions 

My assumptions are as follows: 

a. All expenditures on the hardware and software components of 
MarineNet for the active duty force are considered sunk costs, as the 
money has already been allocated and spent. The system is in place, 
and may be considered fully operational. 

b. I foimd the RDC at MCCES to be fully manned and operational, and 
am therefore including course conversion costs as sunk costs. 

c. Due to time constraints on the scope of this research, the impact of 
MarineNet and DL technologies on the Reserve component will not be 
considered. 

d. Based on my observations at MCCES, I will only consider DL 
applications to advanced-skill level training. 

e. I will apply the industry standard of an estimated 30 percent reduction 
in training time due to the application of DL technologies. 

f. Based on discussions with schoolhouse personnel, current course 
reductions did not include calculated savings based on the use of DL 
technologies. 

2. Determine Alternatives 

The alternative to incorporating the proposed DL initiatives would obviously be 
the status quo, since it would make no sense to dismantle the system that is currently in 
place. The Marine Corps has committed itself to the use of MarineNet, and has no choice 
but to continue down that path. To start from scratch on a completely new program 
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would result in a net loss on all previous network investment. The remaining question is 
how DL technologies may be applied to the advances already in place to realize 
additional benefits in training and education. Although considerable reductions in course 
lengths have already taken place, these have been instituted without adequate 
consideration of the use of DL technologies. The focus up to this point has been more on 
increased classroom efficiency through automation, rather than improvements in 
preassessment and validation techniques. 

3. Determine Costs and Military Value 

My initial computations address only cost reductions that have been attributed to 
CBT, but later consider the additional gains that may be realized by the inclusion of DL 
applications. I also attempt to focus on a more intangible aspect of these changes, 
involving the impact upon the military value of the instruction. Table 5.2 represents the 
current reductions that have already taken place in the reviewed courses. The figures in 
the Training Input Plan (TIP) column represent original course lengths prior to the 
implemented reductions that are reflected in the schoolhouse CDDs. The data were 
compiled from a series of interviews with personnel at T&E Division, HQMC, as well as 
instructors from MCCES. 


Course 

Identification 

COURSE LENGTH (Days) 

30% 

Actual 

Name 

Number 

TIP 

(Old) 


Reduction (Days) 

Reduction (%) 

EEC 

M092721 

93 

40 

65 

57 

TTC 

M09TA31 

112 

80 

78 

29 

BFWC 

M092471 

42 

17 

29 

60 

OCCC 

M0925A1 

118 

60 

83 

49 

CSCC 

M09CHK1 

86 

45 

60 

48 


Table 5.2 Current Reductions in Selected Courses at MCCES 


Based on the figures in the above table, it can clearly be seen that the current reductions, 
which are attributable to the inclusion of CBT and improvements in instructional 
efficiency, have already exceeded the estimated industry standard of 30 percent that is 
associated with the application of DL technologies. Unfortunately no DL applications 
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have yet been instituted, leaving one to wonder if further reductions are feasible. Based 
on discussions with instructors at MCCES, many of these current reductions, directed by 
HQMC, were excessive, arbitrary, and sacrificed skills critical to the quality of the 
training. They feel that fiirther reductions could prove imprudent. Examined at face 
value, one can only wonder how courses of instruction as technical and complex as the 
ones reviewed above can be cut so drastically and still produce a qualified Marine. This 
leads back to the concept of estimating military value. Are these current course 
reductions based solely on cost reduction/avoidance, or is due consideration being given 
to the finished product (i.e., the skill level of the Marine?) According to persoimel at 
MCCES, such is not the case. Arbitrary reductions combined with failure to follow the 
provisions of the SAT and TEMI have been the norm rather than the exception, 
significantly impacting on the military value of the courses. 

Table 5.3 represents data collected fi-om both T&E Division, DL Branch, as well 
as from instructors at MCCES. The model is an exact replica of one used by T&E 
Division to calculate potential savings based on the application of DL technologies. I 
was not able to transcribe the data directly, however, as the figures used for the course 
lengths were in some cases outdated. The course lengths used in my table come directly 
from the CDDs at MCCES. The number of Man-Days is calculated by multiplying the 
yearly course input by the course length. Based on the fact that there is no single, proven 
formula for determining the cost of training a Marine per day, and also in consideration 
that many military units use outdated/unknown cost factors, I analyzed the data from 
T&E Division to see what I could find. Interestingly enough, I found that by dividing the 
total cost of each course after incorporating the 30 percent reduction, by the number of 
new Man-Days (which also included the 30 percent reduction), I came up with a constant 
dollar figure of $82.00 per day. This value remained approximate for all 63 MOSs within 
the data set. 
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Course 

Yearly Input 

Current Length 
(Days) 

Annual 

Man-Days 

Man-Day Cost 

Annual Course 
Cost (Man-Days) 

EEC 

1356 

40 

54,240 

$82 

$4,447,680 

TCC 

188 

80 

15,040 

$82 

$1,233,280 







BFWC 

596 

17 

10,132 

$82 



124 

60 

7440 

$82 

$610,080 


230 

45 

10,350 

$82 

$848,700 


Table 5.3 Annual Cost Computations of Selected Courses at MCCES 


Based on these figures, Table 5.4 illustrates the additional reductions that could be 
realized if the potential savings associated with DL applications were applied. Potential 
savings from DL applications will not be applied to personnel in entry-level courses, but 
instead will be considered for individuals in MAT status. 


Course 

Current Man- 
Days 

Annual Course 
Cost 

(Man-Days) 

New Man-Days 
@ 30% 
Reduction 

New Course 
Cost @ 30% 
Reduction 

Annual 

Savings 

BEC 

54,240 

$4,447,680 




TTC 

15,040 

$1,233,280 

10,528 

$863,296 

$369,984 







BFWC 

10,132 

$830,824 




OCCC 

7440 

$610,080 

5208 

$427,056 

$183,024 

CSCC 

10,350 

$848,700 

7245 

$594,090 

$254,610 


Table 5.4 Cost Analysis Savings of Selected Courses at MCCES based on use of DL Technology 


The calculations are based on a simple reduction of the current number of Man-Days for 
the appropriate course based on the industry standard of 30 percent. This revised figure is 
then multiplied by the $82/day cost per student-day value to attain the new course cost, 
which is then subtracted from the old cost to obtain the aimual savings. 


4. Compare and Select Alternatives 


As seen in the preceding figures, a comparison of the status quo and revised 
methods of instruction illustrates that savings in overall course costs can be realized 
through the implementation of DL technologies. The infrastructure already exists and the 
personnel are in place to convert portions of these courses to a DL format. The key issue 
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here is to ensure that the proper tasks that easily lend themselves to conversion are the 
ones that are converted, while the core competencies critical to the schoolhouse 
environment remain part of the conventional course. To properly accomplish this will 
entail the participation and agreement of SMEs both from the schoolhouse and 
operational units. To do otherwise will worsen the current situation in which course 
reductions have been arbitrary and drastic, with the result of inadequately trained 
Marines. 


5. Conduct Sensitivity Analysis 

A properly conducted sensitivity analysis would entail picking the right variables 
to vary for effect on outcomes. It would involve a review of all tasks and skills 
associated with each individual course by SMEs. More tasks, while lending themselves 
to a DL environment would allow for greater reductions in course length, conversely 
more tasks requiring personal interaction would lengthen or at least maintain course 
curricula at current levels. Once again, the critical issue here is adequate participation by 
SMEs from both the schoolhouse and operational units, who follow the precepts of the 
SAT and TEMI. A fiirther consideration is that the time savings realized from DL 
technologies are not immediately translated into proportional reductions in classroom 
time, without first ensuring that all critical tasks and skills are being taught in one form or 
another. The significant reductions in some of the courses discussed earlier are evidence 
that insufficient consideration was given to the final ability level of course graduates. 

Another variable that could be changed as a part of this analysis would be student 
throughput, based on a review of the TIP. The TIP is produced in relation to a stated 
Fiscal Year (FY), and covers one year from execution and four “out-years” for planning. 
The number of students predicted for each particular course are based on validated 
training requirements for each fiscal year, necessary to obtain or maintain desired 
manning levels (TIP User’s Guide, 1999). This value is thus an important input to the 
TIP process that is the basis for manpower and budgeting costs. If this value is 
overestimated, funds may be over-obligated and not used. If underestimated, special 
efforts are necessary to fund and obtain unplanned school seats. Personnel responsible 
for these estimates (i.e., SMEs and Occupational Field Managers) are therefore extremely 
conscientious and meticulous in their calculations. These values thus remain constant 
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and are not subject to significant change, as seen in the close approximation between the 
courses I observed and the corresponding estimations in the TIP. 

A final variable that could be changed is the estimated cost per student-day. This 
value has been found to be quite nebulous, and in many cases totally unrelated to the 
situation for which it is being used. 

E. ROI MODEL 

Table 5.5 illustrates the ROI process model used by the ASTD (1998) where 
sequential steps simplify a potentially complicated method. This ROI model provides a 
systematic approach to ROI calculations. 



This step-by-step approach keeps the process manageable so that users can tackle one 
issue at a time. The model also emphasizes that this is a logical process that flows from 
one step to another. Applying the model from one ROI calculation to another provides 
consistency, understanding, and credibility (ASTD, 1998). 

Data collection is obviously the starting point and most central aspect of the ROI 
process. As illustrated in Table 5.5, items in the circles are issues that must be addressed 












when determining the specific data collection method. This consideration proved 
particularly challenging for my research as no data supporting the use of DL technology 
currently exist. As such, those tangible factors discussed in Chapter III were considered 
first. As indicated in my assumptions, I considered course conversion costs, support 
costs and resource requirements as sunk costs since the MarineNet infirastmcture is 
already in place and operational at MCESS and MCAGCC. The fimds allocated for this 
project have already been spent and are therefore not a consideration in the effects of DL 
technology applications. Easily quantifiable factors that do impact on a cost analysis 
include timesavings, reduced travel costs, and increased course efficiency. Based on a 
study conducted by Bowes (1991) on the average costs of training first-term Marines, 
Table 5.6 lists values for these factors and combines them as Operation and Maintenance 
(O&M) costs, student pay, and costs of military instructors. 


Course 

Direct 

Costs 

Indirect 

Costs 

Student 

Pay 

Cost of Student 
per Course 

Cost of Instructor 
Time/Course 

Total Cost of 
Course 

TTC 

$1256 

$547 

$11516 

$13319 

$2616 


CSCC 

$631 

$368 

$6380 

$7379 

$6670 

$14,049 

OSCC 

$1313 

$768 

$13,359 

r $15,440 

$6045 

$21,485 


Table 5,6 Training Costs for Individual Marines from Selected Courses at MCCES, from Bowes 
(1991) 


Based on these total cost values (in 1991 dollars) and the predicted benefits listed in 
Table 5.4, a benefit cost ratio (BCR) and a figure for ROI can now be calculated for each 
of the selected courses. Table 5.7 presents these values based on the following formulas: 


BCR = Total Benefits 
Program Costs 


ROI = Total Benefits — Program Costs 
Program Costs 


Course 

Total Projected Annual 
Benefits per Course 

Total Annual 
Course Costs 

BCR 

ROI (%) 

TTC 

$369,984 

$159,350 

2.32 

132% 

CSCC 

$254,610 

$112,392 

2.27 

127% 

OSCC 

$183,024 

$64,455 

2.84 

184% 


Table 5.7 BCR and ROI Calculations on Selected Courses from MCCES Based on DL Technologies 


Based on the above computations, for every dollar invested in DL applications for the 
TTC, $2.32 would be returned. Likewise values of $2.27 and $2.84 could be recognized 
for every dollar of DL technology invested in the CSCC and OSCC respectively. In 
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terms of the ROI calculation, for every dollar of DL technology invested, the dollar 
would be recovered and another $1.32, $1.27, and $1.84 would be produced for the TTC, 
CSCC, and OSCC, respectively. When extrapolated over several years and applied to 
other appropriate courses, the benefits/cost savings of DL technologies become even 
more significant. Unfortunately further research into those specifics is beyond the scope 
of this paper, but definitely merit further research. 

1. Value-added Activities 

Another significant, yet somewhat intangible factor that merits further 
consideration is the impact of DL technology on unit readiness. Unit readiness is 
currently measured by the Status of Resources and Training Systems (SORTS) report 
(Marine Corps Manpower System Brief, 1998). The SORTS report has three principal 
indexes comprised of personnel readiness, equipment readiness, and training readiness. 
The first two categories are relatively easy to measure in terms of readiness. Either a imit 
has the authorized numbers of persoimel and equipment or it doesn’t. Either the assigned 
equipment works or it doesn’t. Accessing the training readiness of the assigned personnel 
is another matter entirely. Although it can readily be determined whether the assigned 
personnel have been to the requisite schools for their rank and billet, it is much more 
difficult to determine the amount of knowledge and skills retained fi-om such training. 

DL has enormous potential in this area. In terms of unit readiness, you would have more 
senior personnel in the operational units for more of their obligated time, who now have 
the same, if not better, opportunities to receive instruction on the most current 
information put out by the schoolhouse. Moreover, when the time does arrive for those 
senior personnel to attend a formal school, they can refresh many of their basic skills 
prior to attending school while still at their parent units. The result of DL applications 
will be a much more informed and up to date student who can proceed through the formal 
instruction at a much faster rate, with greater comprehension and retention. Finally, for 
those personnel unfortunate enough not to be chosen to attend formal school, DL 
technologies will allow them to stay current on the most recent changes in their respective 
MOS. From this perspective, DL technologies can be further viewed as a value-added 
activity that although hard to quantify, improves the overall quality and ability of the 
learner. 
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2. Indirect Benefits 


Several of the less tangible benefits discussed by Mattoon (1998) equally apply. 
Based on observations and discussions at MCCES, DL technologies are favorably 
disposed toward the factors of acceleration, automation, availability, generalizability, 
longevity (through reinforcement), stability, strategics, and transfer. Although difficult to 
quantify, indirect benefits such as these warrant consideration when assessing the total 
value that DL technologies can make to training and education programs. Furthermore, 
the three-level MOE model used by CNET is equally applicable here and would most 
definitely provide a favorable return on DL applications. As supported by observations at 
MCCES, the timesavings MOE has already occurred through the use of the AEC in 
which substantial reductions in attrition and setbacks were evidenced in the LABVOLT 
example. Follow-on planning is already underway to apply the lessons of this successful 
experiment to a DL environment that may be exploited by the reserve component 
(Herring, 2000). 

Phillips (1997) states that intangible measures are the benefits or detriments 
directly linked to the training program, which cannot or should not be converted to 
monetary values. He maintains that these measures are often monitored after the training 
program has been conducted and, although not converted to monetary values, are still 
very important in the evaluation process. A variety of available indirect benefits reflect 
the success of any training and education program. Although they may not be perceived 
as valuable as specific monetary measures, they nevertheless are an important part of an 
overall evaluation. Phillips (1997) further suggests that intangible measures should be 
identified, explored, examined, and monitored for changes when they are linked to the 
program. Collectively, he concludes, they add a unique dimension to the overall program 
results since most, if not all, programs have intangible measures associated with them. 

F. CHAPTER SUMMARY 

According to Derryberry (1998), many organizations are beginning to require 
inclusion of a CBA or ROI analysis to justify an expenditure such as what is required for 
a performance improvement environment or electronic learning system. Simply stated, 
ROI methods are used to demonstrate that the value of the benefits realized from using 
such a solution is going to exceed the price of its development and implementation. She 
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further implies that ROI and technology implementations clearly go hand in hand. 
Organizations are thus looking for new ways to leverage existing and anticipated 
technology investments. Consequently, it is important to remember that in applied 
settings, ROI is particularly valuable as a means to the end of making effective business 
decisions. In applied settings, research-styled methodological rigor is less the point than 
developing a well-supported rationale for ensuring that resources are being allocated in 
economical ways (Derryberry, 1998). 

Even so, Derrybeny (1998) continues, ROI is not simply evaluation methodology 
focused solely on financial concerns. Technology decisions are often not just about the 
financial bottom line. Even if data suggest that a low-end non-technological solution 
may yield positive results, there may be a variety of organizational reasons that this may 
not be as attractive to an organizational funder as is a more sophisticated, costly 
technology solution (Derryberry, 1998). She concludes that while ROI methods provide 
structure for systematically collecting, organizing and compiling data, the interpretation 
of those data, and the resulting perception of value those data represent, may be highly 
subjective. 

Phillips (1997) suggests that ROI is a fifth-level addition to Kirkpatrick’s four- 
level evaluation model (1998). He further suggests that instructional designers who have 
conducted program evaluation using Kirkpatrick’s model may notice strong parallels 
between this expanded view of ROI analysis and each level of Kurkpatrick’s evaluation 
model. Indeed, the design of the front-end ROI analysis and the program evaluation can 
and should be conceptualized simultaneously so that inputs and outputs to the assessment 
and evaluation efforts are consistent (Phillips, 1997). In all cases, special care must be 
given to ensure that changes or values that are being attributed to the performance 
improvement environment are not, in fact, the result of some other influence 
(Lachenmaier & Moor, 1997). 

This chapter reviewed the CBA process with a focus on the basic steps of a 
thorough analysis. Discussion centered on concerns associated with estimating military 
value and the specific application of a CEA to evaluating military versus public sector 
programs. This was followed by a review of intangible, value-added activities that are 
difficult to quantify, but nonetheless important to any analysis of training and education 
programs. A presentation of the analysis strategy outlined the data collected fi'om 
MCCES and provided a foundation for the ROI model. The ROI process model 
attempted to demonstrate the potential benefits of incorporating DL technology into the 
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selected courses based on commonly used BCR and ROI formulas. These calculations 
were followed by a discussion of some additional significant, yet hard to quantify 
variables that should be considered as part of the total analysis. Before making a final 
selection of instructional media, therefore, it is important to determine if the potential 
benefits accrued by DL or technology insertion outweigh the potential costs, or whether 
the instmction will be cost-effective and provide an acceptable Return On Investment. 
The purpose of a cost analysis is to provide the necessary information to facilitate and 
improve the evaluation and decision-making processes. It is important, therefore, that an 
accurate and comprehensive analysis of costbenefit data be accomplished. 
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VI. FINDINGS 


A. FINDINGS FROM T&E DIVISION, DL BRANCH 


1. Funding Support 


Establishing a viable DL program for the USMC is a resource-intensive project 
that has required a significant investment in order to build the necessary infi-astructure 
enabling access for all Marines and Marine Corps civilian personnel. The Marine Corps 
DL program has already received initial program fimding starting in FY99 to establish a 
representative architecture and complete the pilot initiative (HQMC T&E DL 
ROADMAP, 1999). Moreover, the DL program is also fully funded for the FYOl-05 
time frame (HQMC T&E POM-00). 

Although the funding issue, which normally represents the biggest hurdle to a 
program’s implementation has been successfully cleared, the Marine Corps DL program 
is far from fully functional. A number of concerns still remain that must be addressed if 
the true potential of DL is to be fully recognized by the Marine Corps. To help reduce 
the overall cost of the program, several on-going initiatives and programs are being 
leveraged including (HQMC T&E DL ROADMAP, 1999): 

• Base Telecommunications Infrastructure (BTI) - This initiative upgrades the 
telecommunications network infrastructure aboard every base and station 
between FY97 and FY03. The BTI is already fully funded. 

• Reserve Information Network - The R-Net provides telecommunications 
connectivity to each Marine Corps Reserve site. Total R-Net installation has 
been completed. 

• Marine Corps Satellite Education Network - MCSEN is an existing video 
conferencing network available aboard major Marine Corps bases and 
stations. MCSEN is currently devoted to voluntary off-duty education and the 
academic skills program. MCSEN is fully funded. 

• Total Army Distance Learning Program - The TADLP will provide the 
Marine Corps with additional DL courseware and training opportunities, 
including access through the Army’s Doctrine and Training Digital Library 
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(ADTDL), the Training Network (TNET), and Satellite Education Network 
(SEN). The TADLP is funded. 

• National Guard Bureau’s Distributed Training Technology Program (DTTP) - 
The DTTP establishes a robust DL infrastructure across all 50 states and is a 
funded initiative. The Marine Corps and NGB are currently exploring 
collaboration opportunities for the future. 

• Navy DL Programs - CNET is providing assistance with IMI development 
and coimectivity through the CNET’s CESN. The Navy DL programs are 
fully funded. 

Taken collectively, each of these initiatives provide a broad foundation of 
integrated networks and courseware upon which the DL program can interact and expand 
as needed to reach target audiences. Although not specifically addressed in this study, the 
cost savings recognized from the exploitation of these systems alone would prove quite 
substantial since the hardware and supporting infrastructure are already in place. 
Expanding upon the potential of these established systems rather than starting from 
scratch simply makes good business sense. Based on my study of the training 
environment at MCCES, however, I was unable to further access the potential value of 
incorporating these systems into the USMC DL effort as the DL program is still in the 
pilot stage and no further work has been done at this time regarding these issues. 


2. Funding Assumptions 


In addition to the potential savings offered by the use of the above programs, 
justification for DL program funding was based upon four basic assumptions (HQMC 
T&E DL ROADMAP, 1999): 

1) The TEMI, of which DL is a part, would successfully reduce institutional 
training time by approximately 30 percent. 

2) Known as training buy-back, this training time strategy will reduce traditional 
resident training tracks thereby reducing the T2P2 accoxmt. Marines who 
finish training sooner can then be sent to the operating forces faster, thus 
improving operational readiness. 

3) As a result of shorter resident training tracks, the frequency of instruction and 
throughput numbers can be increased accordingly, thus reducing the number 
of Marines awaiting training. 
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4) Increased use of DL will reduce TAD expenditures associated with formal 
training by approximately 30 percent. 

B. FINDINGS FROM MCCES 

Although by no means totally inclusive, these results represent major issues of 
concern to the instructors and administrators at MCCES and MCAGCC. Specific 
findings include: 

1. Reductions in observed course lengths, mandated by HQMC and chiefly 
attributable to CBT, have already taken place at MCCES. These reductions 
are far in excess of the industry standard of 30 percent normally associated 
with DL technology. These reductions were viewed by MCCES personnel as 
excessive, arbitrary, and not in accordance with the provisions of the SAT 
and TEMI. 

2. Although observed course lengths have been systematically reduced and 
Marines are getting to operational units faster, the quality of those Marines’ 
training has come into serious question by both the schoolhouse and the FMF. 

3. Senior instmctors tolerated CBT, but felt it was not a suitable replacement for 
conventional training methods. Their view was that the overall quality of 
instruction was hampered by reductions in conventional student-instructor 
interaction methods. Junior instructors favorably endorsed CBT as the best 
solution to the problem of reduced instructional hours. All persormel 
interviewed in both groups had been at MCCES prior to the activation of the 
AEC. Both groups were in agreement that the imposed reductions in the 
curricula were excessive and sacrificed critical skills necessary to maintain 
quality instmction. 

4. Although no DL applications have been implemented beyond the pilot stage, 
the complete hardware/soflware infrastructure (minus content, which will be 
discussed below) is in place according to plan. 

5. The LAB VOLT experiment, an automated self-paced instructional system in 
electronic theory, considerably reduced the overall student 
recycle/remediation rate from 46 percent to 6 percent. Under the previous 
traditional lecture method, approximately 80 percent of the students were set 
back at least once during the course of instruction. Although originally 
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designed as a stand-alone system, LAB VOLT has been found to work most 
effectively when proceeded by a traditional lecture period. Based on 
empirical evidence, it was found that no additional training time was needed 
to incorporate lecture back into the overall curricula, as the students 
progressed through the self-paced lesson at an increased rate, and with greater 
comprehension when the instruction began with a short lecture period. While 
not directly related to the DL program, these impressive results definitely 
have technology-related implications such as increased comprehension and 
completion rates. 

6. It was the unanimous opinion of all personnel interviewed at MCCES, that 
with the possible exception of MAT, DL technology had no application to 
entry-level skill training. This was attributed to the fact that Marines at this 
level of training are most receptive to and in need of the personal interaction 
techniques of conventional classroom instruction. Historical evidence 
suggests that Marines, at this point in their enlistment, are often overwhelmed 
by the dramatic change in their environment and life style. Attendance at 
their formal MOS school represents the first real independence new Marines 
are given following the rigors of basic training and MCT. To expect these 
young men and women to sacrifice the little free time they are given in 
pursuit of additional academic challenges is somewhat unreahstic. 
Furthermore, any type of non-MOS instruction outside of formal training was 
viewed as a distraction fi'om the Marine’s primary duty of excelling in his 
class. In support of the Marine’s transformation process, and in the interest of 
imbuing the ideals of teamwork and imit cohesion, formal schoolhouse 
training is therefore considered a must for entry-level training. 

7. Company A introduced a maximum training effort to eliminate its MAT 
problem and realized limited success. Their solution involved increasing the 
training tempo to 24 hours per day. Problems quickly surfaced that included 
such issues as equipment failure fi’om ovemse, and instractor fatigue. It was 
discovered that the problem was never really solved, only pushed further 
down the training pipeline to succeeding phases of training that could not 
handle the increased throughput. Through a process of trial and error the 
training tempo was eventually reduced to current levels, which are considered 
optimal. 
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8. Although significantly reduced, the MAT problem still existed for Company 
A. In response, Company A trains their personnel in a MAT status in other 
non-MOS essential, yet time consuming-functions such as drill and 
ceremonies, imiform inspections, physical fitness, etc. Although considered 
an efficient solution to the MAT problem, this approach has not yet been 
standardized among the other training companies at MCCES. 

9. Personnel interviewed at MCCES generally agreed that individuals in a MAT 
status would be good candidates for DL technology applications. The time 
would be advantageous for introductory lessons in support of the 
transformation process such as personal finance, map reading, and other 
general knowledge courses. There would be no distraction from formal 
training since it had not yet begun, so the Marines could devote their full 
attention to the instruction. Once formal MOS related training began, 
however, it was strongly emphasized that all DL instruction should stop so 
the Marines could devote their full time and attention toward that effort. 

10. Advanced-skill level courses were deemed suitable for DL applications. 
Distance Learning could aid in refreshing the basic skills of senior Marines 
prior to formal school attendance. This action would not only allow the 
Marines to begin the course of instruction at a higher skill level, but would 
contribute to faster progression and comprehension rates, leading to possible 
course reduction. Simply stated, preassessment and course compression 
would be enhanced. 

11. Instructors at MCCES felt that CBT and DL technologies should not be used 
as a blanket excuse to automatically reduce the length of courses. The 
Marine Corps is continually fielding newer and more complex equipment as 
compliments to, rather than replacements for, existing equipment. As such. 
Marines are expected to be intimately familiar with complex networks of 
technologically sophisticated equipment. Where and when are Marines to be 
taught how to operate and maintain this increasing inventory of equipment? 
MCCES personnel suggest that training timesavings realized from CBT and 
DL technologies should not be automatically eliminated from curricula, but 
should be maintained for inclusion of additional necessary information. 

12. MCCES uses inaccurate costing data. The cost of an individual Marine 
training day, estimated at $271, had no supporting historical proof No one at 
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MCCES could explain where this value came from or under what 
assumptions it had originally been calculated. 

13. Based on my assumptions, I calculated the cost of a Marine training day to be 
approximately $82.00. Combining this value with previously calculated data 
on the cost of training first-term Marines, I was able to calculate a BCR and 
ROI for three observed courses at MCCES. Results were favorable with a 
positive ROI that averaged approximately 147 percent for the three selected 
courses, or an average of $1.47 returned for every dollar invested. Elements 
of a CEA were also included in my analysis, which indicated potential 
favorable results for DL applications in terms of the military value added. 
Distance Learning technology has the potential to keep senior Marines in 
operational units for a longer period of time, while affording them the same 
opportunities for learning that school trained individuals currently have. 
Distance Learning applications can thus be viewed as a value added activity 
that enhances the cost effectiveness of a training program and promotes unit 
readiness. 

14. SMEs chosen for CCRBs are currently chosen on the basis of seniority, and 
are not necessarily the most experienced individuals in their field. In light of 
the rapid rate at which modem technology continues to advance, many 
systems are fielded that the most senior personnel in a unit know little or 
nothing about. These same individuals, however, are the ones who are 
chosen, as SMEs, to make decisions concerning which critical skills 
(regarding these new systems) will be taught at the schoolhouse and which 
will be handled in the FMF. They obviously push for the schoolhouse to 
conduct the majority of the instraction. As expected, a dilemma ensues when 
the responsibility is placed on the schoolhouse to teach an increasing number 
of these critical skills, while simultaneously the length of the curricula is 
disproportionately reduced. 

15. A review of MOE regarding CBT (i.e., test scores), from three random 
classes prior to the activation of the ABC and three random classes after, 
found no significant difference in the performance of the students. These 
findings applied to both entry-level and advanced-level training courses 
observed at MCCES. 
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16. A fully operational VTT facility was located at MCAGCC, but no plans had 
been designed to exploit its potential as part of a DL program. 

17. The LRC, although fully operational at both MCCES and MCAGCC, was 
significantly underused due to a lack of content. The number of automated 
courses currently available totaled only three, and failed to generate sufficient 
interest due to software compatibility problems as well as difficulties with 
electronic testing procedures. MCI, which has total responsibility for course 
conversion procedures, has yet to move beyond the initial pilot stage of the 
program. 

18. Insufficient historical data regarding the USMC DL program currently exist 
to determine the actual contribution DL can make to the Marine Corps. This 
is due to the fact that the program has not yet proceeded beyond the initial 
pilot stage, although, as mentioned, the supporting infrastructure is in place 
and ready for use. 
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VIL CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

1. DL Program Funding Assumptions 

The previously discussed funding assumptions, upon which DL program was 
justified, represent the logic behind the entire process. I will therefore focus my 
discussion aroxmd these four main points. 

a. Training Time Reduction 

The industry standard of a 30 percent reduction in conventional training 
time, through the use of DL technology, represents a sound benchmark for a cost 
analysis, but this figure is by no means always dependable. It was found that all of the 
courses reviewed at MCCES had already been reduced by more than 30 percent due to 
the introduction of CBT, as well as improvements in instructional efficiency (Taylor, 
2000). Some of these reductions were viewed as capricious and excessive, impacting on 
the overall quality of the course and the ability/skill level of the graduates. Although my 
own analysis was based on a similar standard of 30 percent and produced favorable 
results, the issue remains questionable until DL technology is actually implemented and 
hard data, in terms of DL course enrollment and completion rates, become available. 

In some cases, further reductions in course length of any kind could 
eliminate crucial skills that cannot be successfully learned other than through face-to-face 
interaction. Evidence has been presented of deficiencies in the skill levels of new 
Marines leaving MCCES that must be taught in the operational units. Many of these 
difficulties have been traced to miscommimication between SMEs at the schoolhouse and 
in the FMF over critical ITS, and responsibility for teaching them. More importantly, the 
spirit of the SAT and TEMI, which comprise the foundation of curricula development 
procedures for the Marine Corps (and were not followed at MCCES), should be more 
closely adhered to when converting course content to a DL format. This should help 
ensure that similar mistakes are not repeated with the DL program, and critical skills most 
suitable for a formal classroom environment are not eliminated in favor of faster 
completion rates. 
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Once sufficient content is available for practical use of DL technology, the 
Marine Corps needs to promote this form of training and education with the utmost 
enthusiasm. Failure to gain the support of senior personnel, both within the fonnal 
school system as well as the FMF, could seriously affect buy-in of the DL program 
among the rank and file. Linking the use of DL programs with increased advancement 
opportunities is but a first step toward promoting the acceptance and use of this training 
process. 

In a similar vein. Marines no longer have the luxury to wait until afforded 
the opportunity to attend a formal school and expect the faculty to provide all the 
necessary information for proficiency in a relatively limited period of time. Familiarity 
with elementary computer skills and basic MOS knowledge must already be current prior 
to attending school. The bxirdensome operational tempo most Marines face in today’s 
environment certainly does not provide the individual with the time necessary to acquire 
many of these skills, prior to attending formal training, yet this situation by no means 
relieves the individual of the responsibility for knowing them. This circiunstance is not 
much different than that faced by civilian coimteiparts who are continually required to 
refresh their skills to stay competitive, but are reluctant to take time from their jobs to do 
so. The individuals who successfully accomplish this formidable task are the ones who 
can most efficiently manage their time. Distance Learning technologies can significantly 
affect one’s ability to manage time more efficiently for training purposes. 

Distance Learning programs can also greatly assist in eliminating the 
unfamiliarity problem that many senior Marines experience with fielding new equipment 
in the FMF. Those senior Marines who are expected to be the most knowledgeable in 
regard to new systems, and yet have the least opportunity to attend formal school, can use 
DL technology to maintain their MOS skills. A value added consideration here is that 
DL technology can provide badly needed expertise to senior Marines, who may find 
themselves chosen as SMEs, and must recommend which skills to teach at school and 
which to handle in the FMF. Familiarity with DL programs will thus allow SMEs to 
make better-informed decisions regarding training and education programs. 

b. Training Buy-back 

The issue of training buy-back and its effect on the T2P2 account is also 
somewhat questionable. The previous CMC specifically directed MCCES and all other 


80 



foraial Marine Corps schools to reduce the length of courses to get Marines to the 
operating forces faster and thus improve operational readiness. The first part of this 
guidance was readily achieved, but the impact on operational readiness remains in doubt 
for reasons previously discussed. Marines are, in fact, getting to operational units faster, 
but the quality of their training has come into serious question. Based on my research at 
MCCES, this problem was again traced to an improper needs assessment by SMEs, who 
failed to follow the procedures of the SAT and TEMI. Whether DL initiatives will 
positively affect training buy-back depends upon a thorough analysis by qualified SMEs 
of those critical skills that must remain in the schoolhouse, and those that may be applied 
in a DL environment. 

Short of total mobilization for war, the Marine Corps continually operates 
with a Table of Organization (T/0) well below its authorized manning level. Although 
they may not like it, unit commanders have become accustomed to this situation and in 
most circumstances manage to meet their obligations with the personnel and equipment 
that they have. As such, the FMF does not miss the Marines it has not yet received from 
the training pipeline. Efforts to move entry-level Marines through the training cycle more 
quickly is a laudable goal, but only if the quality of the training those Marines receive is 
not sacrificed for the sake of an increased rate of throughput. It does no good to get a 
Marine to the FMF more quickly if that Marine can’t satisfactorily perform his job. 
Furthermore, unit readiness suffers as training priorities must now be changed to teach 
the new Marine the skills he should have learned in school. Evidence gathered fi'om 
MCCES indicates that this has been the case in some instances. 

The success of the LAB VOLT experiment, although not directly related to 
DL technology, demonstrates the potential of automation to increase the efficiency of 
training and education programs. Distance Learning for the Marine Corps is by no means 
viewed as a panacea for the ailments of traditional classroom instruction, but rather as a 
complement that will serve to benefit all Marines. Just as the LABVOLT system was 
found to function most efficiently when combined with limited, traditional lecture 
penods, DL will provide the greatest benefit when properly combined with current 
training methods. 

In a rush to apply the timesaving benefits of CBT and DL technologies, 
the Mannes Corps needs to consider the warnings of the personnel tasked with working 
with these technologies. With the increased amount of information that all Marines are 
responsible for in today’s technologically advanced environment, formal schools find 
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themselves taxed to teach more in less time. Automated technologies can certainly assist 
in this effort, but the advantages gained through the technology can quickly disappear if 
there is insufficient time to teach vital information. Compromises will have to be made 
on both sides of the issue, but the Marine Corps needs to ensure that training and 
education programs are optimized to meet the needs of the individual Marine and are not 
solely focused on increased throughput. 

c. Marines Awaiting Training 

Based on initiatives imdertaken by instructors at MCCES, the issue of 
MAT has been addressed with mixed results. Placing Marines in a training hiatus once 
their course had begim was found to have adverse effects on the retention of knowledge 
learned from earlier phases of training (Herring, 2000). It was therefore determined that it 
was in the best interests of the students to place them in a MAT status prior to the start of 
a course rather than interrupt the training cycle once the POI had begun. The contribution 
that DL can make to this situation is perhaps the only positive area in regard to entry- 
level training. It was the unanimous opinion of all personnel interviewed at MCCES that 
once formal training began for entry-level Marines, DL technology had little application 
due to the need to maintain academic discipline and enhance knowledge transfer among 
new Marines. 


d. TAD Reduction 

The assumption that increased use of DL will reduce TAD expenditures 
associated with formal training by 30 percent is based on industry standard, and will 
remain as such imtil hard data becomes available for the Marine Corps to either support 
or refute this claim. A review of over 33 empirical studies of civilian businesses involved 
with DL over the past decade has found this figure to be quite reliable (Johnston & 
Fletcher, 1997). Although easily quantifiable cost figures such as these are most 
frequently promoted as justification for DL technology, this paper does not suggest that 
the Marine Corps should continue with its implementation plan simply because the 
figures and the technology have pushed it there. Barring any insurmountable obstacles to 
full implementation, however, which this study has not found, the current plan for the DL 
program should be continued. 
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2. Summary 


Based on my observations at MCCES, significant progress has been achieved in 
increasing instructional efficiency through the use of automation in the classroom. When 
properly applied, DL technology will continue this trend with substantial positive returns 
on the investment. A number of hurdles will have to be cleared before any positive ROI 
will be realized. For the Marine Corps, DL technology is not applicable to entry-level 
training, and with the possible exception of personnel in a MAT status, should no longer 
be considered for this category of training. Distance Learning technology does have 
enormous potential in the Marine Corps, however, for advanced skill level training and 
should be implemented to the fullest extent possible as soon as circumstances permit. 
The potential savings that DL can offer through preassessment training and course 
compression have been briefly considered in this paper, and definitely merit further study 
as more data become available. This implementation procedure should not be too hasty, 
as a thorough needs assessment, in accordance with the SAT and TEMI, is critical to 
identify the crucial skills that can be taught most effectively in a formal schoolhouse 
environment. 

B. RECOMMENDATIONS 

INCREASE MCI IN VOLVEMENT . It was observed that the LRCs at MCAGCC 
were currently underutilized for lack of adequate electronic courses of instruction. The 
number of courses available totaled only three, and failed to generate sufficient interest 
due to software compatibility problems as well as difficulties with electronic testing 
procedures. I am confident that once the MCI, which has sole responsibility for this area, 
works out these software problems and fields more courses, the full potential of this DL 
entity will be fully realized. Further research into a cost analysis of the conversion 
program that MCI is currently using, is definitely warranted. 

EXPAND VTT USE. It was also discovered that although MCAGCC has a fully 
functional VTT site, nothing is currently being done to use this technology for DL 
applications. The benefit of such technology has been well documented by Cabrera 
(1999) and rates further study. 
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FOCUS ON THE RESERVE COMPONENT . Additional research is particularly 
needed in regard to DL applications and the reserve component of the Marine Corps. 
Discussions with schoolhouse personnel at MCCES revealed several areas of interest 
where DL was currently being studied for use in training reserve Marines. Marine Forces 
Reserve (MARFORRES) has recently invested considerable funding into a prototype of 
the LABVOLT system for use at selected reserve centers throughout the country 
(Herring, 2000). Distance Learning applications were found to be particularly applicable 
to advanced skill level training where reservists could become proficient on the basics of 
their MOS while at their parent units. Such application would thus permit more efficient 
use of the two-week active duty period in the summer months, when most reservists 
usually attended formal schooling (Foy, 2000). A study was conducted by the Marine 
Corps regarding the potential of DL technology applications for the reserve component 
(Metzko, et.al., 1996), yet based on my research little has been accomplished since. It is 
definitely time for a more current study to see where DL can most significantly contribute 
to the reserve component. 

EXPLORE DL APPLICATIONS TO MAT . If DL is to be used at all for entry- 
level training, the most appropriate time should be for personnel in a MAT status. The 
application of DL technology and MAT definitely warrants further study for potential 
training dividends. 

ENCOURAGE PERSONAL INITIATIVE . The Marine Corps must continue to 
emphasize the importance and necessity for the individual Marine to focus on his own 
educational advancement. As a result of increased training requirements combined with 
reductions in formal school courses, it is incumbent upon the individual Marine to pursue 
education and training on his own initiative more than ever before. Distance Learning 
applications can be of use in this effort. 

TTTTT.T7F. TIMESAVINGS EFFICIENTLY . Refrain from viewing training 
technology solely as a curricula-cutting measure. The timesavings realized from the use 
of CBT and DL technology should not be automatically eliminated from established 
training schedules. Such technology provides the greatest benefit when used to 
complement rather than replace conventional training methods. 

ELIMINATE INSTITUTIONAL BIAS . To fully realize the potential of DL, the 
Marine Corps must deal with several challenges. First, the anticipated resistance to 
change embedded within the institution, although found to be minimal, must be 
overcome. Distance Learning establishes a new learning environment and considerably 


84 



changes established training and education paradigms. Such changes will naturally meet 
resistance due to lack of experience with curriculum development procedures developing 
effective DL solutions, unfamilianty with the technology, and the erroneous perception 
that DL will eliminate jobs or diminish resident instruction. Lack of personal experience 
appears to be the key reservation among detractors of DL programs. The Marine Corps 
must aggressively endorse the value and use of DL technology. 

prom ote personal incentives . To promote effective buy-in of DL 
technology among Marines, a system of incentives for DL programs that is tied to 
promotion or advancement opportunities must be established. Such incentives will most 
likely provide the catalyst for rapid expansion of current DL initiatives. 

PROVIDE GLJIDANCE AND M ENTORSHIP . Providing effective mentoring 
and maintaining cohesion in a distributed training environment will prove challen gin g 
The shift to student-centered learmng must be balanced with a team training approach 
that is supported and supervised by senior leaders. This shift in learning methods will by 
no means occur overnight, but will require several years to implement. Furthermore, not 
every Marine will learn at the same pace using this approach. Some Marines will require 
a more structured learning environment along with increased personal interaction to 
facilitate learning. Any training and education solution that is developed must be flexible 
enough to accommodate alternate learning methods and schedules. The key ingredient to 
this concept will be establishing an environment at the unit level that is conducive to 
learning and providing the proper leadership to ensure that DL programs are successfully 
completed. 

FOLLOW THE SAT AND TEMI . Evidence gathered firom MCCES reveals that 
many of the problems encountered when applying automated training technology resulted 
from a failure of experienced SMEs to properly follow established procedures. These 
publications clearly outline the proper method for developing, changing, and updating 
curricula. Following these procedures when converting existing course materials to a DL 
format, or when developing new curricula, will avoid many of the problems encountered 
in this study. Furthermore, as indicated in these directives, SMEs should be chosen based 
on experience levels, rather seniority alone. A more careful screening of SMEs will 
avoid many of the pitfalls associated with an improper needs assessment, when reviewing 
curricula for DL conversion. 

FURNISH ACCESS TIME. Issues regarding Internet access aboard Marine 
Corps bases and stations must also be addressed and resolved prior to realizing the full 
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benefit of DL. A recent study by the Army Research Institute (ARI, 1998) revealed that 
over 70 percent of the officers interviewed had regular Internet access, while only 30 
percent of the enlisted soldiers interviewed had similar Internet access. Accordingly, the 
Army has identified lack of Internet access for soldiers as a critical barrier to success in 
their DL program. Although the Marine Corps has not yet conducted a similar study, it 
seems logical that similar percentages could be applied to the Marine Corps personnel. In 
order for DL to fully succeed therefore. Marines must have significantly greater access to 
the Marine Corps network and the Internet. 

High operational tempo also makes finding additional time to train and educate 
very difficult. Finding time for Marines and civilian personnel to learn represents 
perhaps the greatest challenge to leaders, trainers and educators in the information age. 
DL programs cannot be relegated to off-duty hours. If the Marine Corps is to realize the 
full potential of new instructional delivery methods, deliberate steps must be taken to 
schedule training and education events into normal duty hours. 

REVIEW COHORT QUALITY & DL . A final consideration of my study regards 
the quality of the Marines attending MCCES. The quality of the training cohorts 
attending MCCES is among the highest in the Marine Corps, as these types of courses 
require higher Armed Forces Qualification Test (AFQT) scores to successfully meet the 
demands of the billets (i.e., to maintain and repair complex electronic and technical 
equipment). As such, it would be beneficial to see if a relationship exists between the 
success rate that Marines experience with DL technology and their level of intelligence. 
It would be interesting to examine the success rates with CBT and DL applications at 
other formal schools within the Marine Corps. Also, with increasing numbers of non- 
high school graduates entering the Marine Corps, it will be equally interesting to see how 
well prepared these Marines of the future will be for an automated training environment, 
particularly one that emphasizes self study through DL technology. Once sufficient data 
from DL applications become available, further research should be conducted in this 
intriguing area of training and education. 

BE PATIENT . Due to the fact that the Marine Corps has yet to actively 
implement any DL programs beyond the initial pilot stage, there is a lack of sufficient 
data to do a more thorough analysis. Although the early results of automation in the 
classroom have been positive regarding ROI and the contribution that DL technology can 
make appears promising, it is clearly much too early to reach any final conclusions. The 
USMC’s DL roadmap for the future must be given sufficient time to be fully 


86 



implemented and allowed to reach fruition before its ultimate utility can be properly 
judged. The issue should therefore be revisited in about three to five years. 
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APPENDIX A. ROI DATA FOR USMC MILITARY OCCUPATIONAL 
SPECIALTIES COMPILED AT HQMC, T&E DIVISION, DL BRANCH, 1999 
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5811 A10RF31 MILITARY POLICE (USMC) 1064 _ ^M _ 67,032 _ 46,922 129 _55 $3,856,636 $1,652,844 5811 

3381 A1433L1 FOOD SERVICE SPECIALIST 1120 _58_ ^ _ 64,960 _ 45,472 125 _53 $3,737,425 $1,601,753 3381 

6412 N23A952 AVIONICS TECHNICIAN I 589 105 74 61,845 43,292 119 51 $3,558,205 $1,524,945 6412 
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APPENDIX B. ABBREVIATIONS AND ACRONYMS 


ACSDE 

American Center for the Study of Distance Education 

ADTDL 

Army Doctrine and Training Digital Library 

AEC 

Automated Electronic Classroom 

AFIT 

Air Force Institute of Technology 

AFQT 

Armed Forces Qualification Test 

ALC 

Area Learning Center 

AOB 

Average on Board 

BFWC 

Basic Field Wireman’s Course 

BTI 

Base Telecommimications Infrastructure 

BSNCOC 

Battle Staff Noncommissioned Officer’s Course 

CBA 

Cost Benefit Analysis 

CBT 

Computer Based Training 

CCRB 

Course Curriculum Review Board 

CDD 

Course Descriptive Data 

CD-ROM 

CD-Read-Only Memory 

CEA 

Cost Effectiveness Analysis 

CESN 

CNET Electronic Schoolhouse Network 

CMC 

Computer-mediated Communications 

CNA 

Center for Naval Analyses 

CNET 

Chief of Naval Education and Training 

CNO 

Chief of Naval Operations 

CSCC 

Commimications Systems Chief Course 

DII 

Defense Information Infi*astructure 

DISN 

Defense Information Systems Network 

DL 

Distance Learning 

DEC 

Distance Learning Center 

DoD 

Department of Defense 

DTTP 

Distributed Training Technology Program 

EO 

Executive Order 

EPSS 

Electronic Performance Support Systems 

ET 

Embedded Training 

FLC 

Functional Learning Center 

FMF 

Fleet Marine Force 

GFE 

Government Furnished Equipment 

GFL 

General Facilitative Links 

IDA 

Institute for Defense Analyses 

IDEA 

Institute for Defense Education and Analyses 

IMI 

Interactive Multimedia Instruction 

IT 

Information Technology 

ITS 

Individual Training Standards 
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ISD 

Instructional Systems Development 

LCCE 

Life Cycle Cost Estimate 

MARFORJRES 

Marine Forces Reserve 

MAT 

Marines Awaiting Training 

MCAGCC 

Marine Corps Air Groimd Combat Center 

MCCES 

Marine Corps Communications and Electronics School 

MCI 

Marine Corps Institute 

MCSC 

Marine Corps System Command 

MCSEN 

Marine Corps Satellite Education Network 

MCT 

Marine Combat Training 

MCTFS 

Marine Corps Total Force System 

MOE 

Measure of Effectiveness 

MOS 

Military Occupational Specialty 

N/MCI 

Navy/Marine Corps Intranet 

NPV 

Net Present Value 

occc 

Operations Communications Chief Course 

ODUSD(R) 

Office of the Deputy under Secretary of Defense (Readiness) 

OUSDP&R 

Office of the Under Secretary of Defense for Personnel and 
Readiness 

PCS 

Permanent Change of Station 

POI 

Program of Instruction 

POM 

Program Objective Memorandum 

R-NET 

Reserve Information Network 

ROI 

Return on Investment 

PME 

Professional Military Education 

R&D 

Research and Development 

RDC 

Regional Development Center 

SAT 

Systems Approach to Training 

SEN 

Satellite Education Network 

SFL 

Specific Facilitative Links 

SME 

Subject-Matter Expert 

SORTS 

Status of Resources and Training Systems 

T2P2 

Transients, Trainees, Patients, and Prisoners 

l&E 

Training and Education 

TAD 

Temporary Additional Duty 

TADLP 

Total Army Distance Learning Plan 

TCP/IP 

Transmission Control Protocol/Intemet Protocol 

TDY 

Temporary Duty 

TEMI 

Training and Modernization Initiative 

TEPOP 

Training and Education Point of Presence 

TIP 

Training Input Plan 

TNET 

Training Network 
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TRADOC 

TTC 

VR 

VTC 

VTT 


Training and Doctrine Command 
Technician Theory Course 
Virtual Reality 
Video Teleconferencing 
Video Teletraining 
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